La classe di analisi html open source non analizza correttamente gli spazi tra i paragrafi

Question

Sto utilizzando un metodo open source che analizza il testo html in una NSString.

Le stringhe risultanti hanno grandi quantità di spazio bianco tra la prima coppia di paragrafi, ma solo una riga di spazio per i paragrafi successivi. Ecco un esempio di output.

Di seguito è riportato il metodo che sto chiamando. Ho solo cambiato due righe del codice. Per stopCharacters e newLineAndWhitespaceCharacters, ho rimosso /n dal set di caratteri perché quando è stato incluso, l'intero testo era un paragrafo lungo.

- (NSString *)stringByConvertingHTMLToPlainText { 

    // Pool 
    NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init]; 

    // Character sets 
    NSCharacterSet *stopCharacters = [NSCharacterSet characterSetWithCharactersInString:[NSString stringWithFormat:@"< \t\r%C%C%C%C", 0x0085, 0x000C, 0x2028, 0x2029]]; 
    NSCharacterSet *newLineAndWhitespaceCharacters = [NSCharacterSet characterSetWithCharactersInString:[NSString stringWithFormat:@" \t\r%C%C%C%C", 0x0085, 0x000C, 0x2028, 0x2029]]; 
    NSCharacterSet *tagNameCharacters = [NSCharacterSet characterSetWithCharactersInString:@"abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ"]; 

    // Scan and find all tags 
    NSMutableString *result = [[NSMutableString alloc] initWithCapacity:self.length]; 
    NSScanner *scanner = [[NSScanner alloc] initWithString:self]; 
    [scanner setCharactersToBeSkipped:nil]; 
    [scanner setCaseSensitive:YES]; 
    NSString *str = nil, *tagName = nil; 
    BOOL dontReplaceTagWithSpace = NO; 
    do { 

     // Scan up to the start of a tag or whitespace 
     if ([scanner scanUpToCharactersFromSet:stopCharacters intoString:&str]) { 
      [result appendString:str]; 
      str = nil; // reset 
     } 

     // Check if we've stopped at a tag/comment or whitespace 
     if ([scanner scanString:@"<" intoString:NULL]) { 

      // Stopped at a comment or tag 
      if ([scanner scanString:@"!--" intoString:NULL]) { 

       // Comment 
       [scanner scanUpToString:@"-->" intoString:NULL]; 
       [scanner scanString:@"-->" intoString:NULL]; 

      } else { 

       // Tag - remove and replace with space unless it's 
       // a closing inline tag then dont replace with a space 
       if ([scanner scanString:@"/" intoString:NULL]) { 

        // Closing tag - replace with space unless it's inline 
        tagName = nil; dontReplaceTagWithSpace = NO; 
        if ([scanner scanCharactersFromSet:tagNameCharacters intoString:&tagName]) { 
         tagName = [tagName lowercaseString]; 
         dontReplaceTagWithSpace = ([tagName isEqualToString:@"a"] || 
                [tagName isEqualToString:@"b"] || 
                [tagName isEqualToString:@"i"] || 
                [tagName isEqualToString:@"q"] || 
                [tagName isEqualToString:@"span"] || 
                [tagName isEqualToString:@"em"] || 
                [tagName isEqualToString:@"strong"] || 
                [tagName isEqualToString:@"cite"] || 
                [tagName isEqualToString:@"abbr"] || 
                [tagName isEqualToString:@"acronym"] || 
                [tagName isEqualToString:@"label"]); 
        } 

        // Replace tag with string unless it was an inline 
        if (!dontReplaceTagWithSpace && result.length > 0 && ![scanner isAtEnd]) [result appendString:@" "]; 

       } 

       // Scan past tag 
       [scanner scanUpToString:@">" intoString:NULL]; 
       [scanner scanString:@">" intoString:NULL]; 

      } 

     } else { 

      // Stopped at whitespace - replace all whitespace and newlines with a space 
      if ([scanner scanCharactersFromSet:newLineAndWhitespaceCharacters intoString:NULL]) { 
       if (result.length > 0 && ![scanner isAtEnd]) [result appendString:@" "]; // Dont append space to beginning or end of result 
      } 

     } 

    } while (![scanner isAtEnd]); 

    // Cleanup 
    [scanner release]; 

    // Decode HTML entities and return 
    NSString *retString = [[result stringByDecodingHTMLEntities] retain]; 
    [result release]; 

    // Drain 
    [pool drain]; 

    // Return 
    return [retString autorelease]; 

} 

EDIT:

Ecco l'NSLog della stringa. Ho incollato solo i primi paragrafi

Mitt Romney spent the past six years running for president. After his loss to President Barack Obama, he'll have to chart a different course. 


His initial plan: spend time with his family. He has five sons and 18 grandchildren, with a 19th on the way. 






"I don't look at postelection to be a time of regrouping. Instead it's a time of forward focus," Romney told reporters aboard his plane Tuesday evening as he returned to Boston after the final campaign stop of his political career. "I have, of course, a family and life important to me, win or lose." 

The most visible member of that family — wife Ann Romney — says neither she nor her husband will seek political office again. 

ecc ....

for (int j = 25; j< 50; j++) { 
    char test = [completeTrimmed characterAtIndex:([completeTrimmed rangeOfString:@"chart a different course."].location + j)]; 

     NSLog(@"%hhd", test); 
    } 

012-11-11 17:15:57.668 LMU_LAL_LAUNCHER[5431:c07] 32 
2012-11-11 17:15:57.669 LMU_LAL_LAUNCHER[5431:c07] 32 
2012-11-11 17:15:57.669 LMU_LAL_LAUNCHER[5431:c07] 10 
2012-11-11 17:15:57.670 LMU_LAL_LAUNCHER[5431:c07] 32 
2012-11-11 17:15:57.670 LMU_LAL_LAUNCHER[5431:c07] 32 
2012-11-11 17:15:57.670 LMU_LAL_LAUNCHER[5431:c07] 10 
2012-11-11 17:15:57.671 LMU_LAL_LAUNCHER[5431:c07] 32 
2012-11-11 17:15:57.671 LMU_LAL_LAUNCHER[5431:c07] 32 
2012-11-11 17:15:57.671 LMU_LAL_LAUNCHER[5431:c07] 10 
2012-11-11 17:15:57.672 LMU_LAL_LAUNCHER[5431:c07] 32 
2012-11-11 17:15:57.672 LMU_LAL_LAUNCHER[5431:c07] 72 
2012-11-11 17:15:57.672 LMU_LAL_LAUNCHER[5431:c07] 105 
2012-11-11 17:15:57.673 LMU_LAL_LAUNCHER[5431:c07] 115 
2012-11-11 17:15:57.673 LMU_LAL_LAUNCHER[5431:c07] 32 
2012-11-11 17:15:57.673 LMU_LAL_LAUNCHER[5431:c07] 105 
2012-11-11 17:15:57.673 LMU_LAL_LAUNCHER[5431:c07] 110 
2012-11-11 17:15:57.674 LMU_LAL_LAUNCHER[5431:c07] 105 
2012-11-11 17:15:57.674 LMU_LAL_LAUNCHER[5431:c07] 116 
2012-11-11 17:15:57.674 LMU_LAL_LAUNCHER[5431:c07] 105 
2012-11-11 17:15:57.675 LMU_LAL_LAUNCHER[5431:c07] 97 
2012-11-11 17:15:57.675 LMU_LAL_LAUNCHER[5431:c07] 108 
2012-11-11 17:15:57.675 LMU_LAL_LAUNCHER[5431:c07] 32 
2012-11-11 17:15:57.675 LMU_LAL_LAUNCHER[5431:c07] 112 
2012-11-11 17:15:57.676 LMU_LAL_LAUNCHER[5431:c07] 108 
2012-11-11 17:15:57.676 LMU_LAL_LAUNCHER[5431:c07] 97 

Answer 1

Ho provato con la domanda di cui sopra, e questo è come ho riparato,

NSString *retString = [[result stringByDecodingHTMLEntities] retain]; 
[result release]; 

retString = [retString stripDuplicateCharactersInSet:[NSCharacterSet whitespaceCharacterSet] withString:@" "]; 
retString = [retString stripDuplicateCharactersInSet:[NSCharacterSet newlineCharacterSet] withString:@"\n"]; 

Ho definito un metodo categoria sulla NSString come,

- (NSString *)stripDuplicateCharactersInSet:(NSCharacterSet *)characterSet withString:(NSString *)joiningString; 

L'attuazione è la seguente,

- (NSString *)stripDuplicateCharactersInSet:(NSCharacterSet *)characterSet withString:(NSString *)joiningString { 

    NSMutableString *originalStr = [NSMutableString string]; 

    if (!self) { 
     return nil; 
    } 

    NSArray *componentsArray = [self componentsSeparatedByCharactersInSet:characterSet]; 

    int counter = 0; 
    for (NSString *stringComponent in componentsArray) { 

     counter ++; 

     if ((stringComponent) && ([stringComponent length] > 0) && (![stringComponent isEqualToString:@" "]) && ((![stringComponent isEqualToString:@"\n"]) || (![joiningString isEqualToString:@"\n"]))) { 

      if ([componentsArray count] == counter) { 
       [originalStr appendFormat:@"%@", stringComponent];     
      } else { 
       [originalStr appendFormat:@"%@%@", stringComponent, joiningString]; 
      } 
     } 
    } 

    return originalStr; 
} 

Aggiungere il metodo sopra nel file NSString+HTML.m come categoria su NSString. Fondamentalmente nel codice HTML fornito da te, spazi vuoti e newline sono stati mescolati più volte, e il tentativo di eliminare la newline da solo non funzionava. Quindi sto rimuovendo le newline duplicate e gli spazi bianchi come mostrato sopra confrontando se la stringa ha newline o spazi dopo lo stripping e quindi accodandosi alla stringa principale.

In alternativa, si può anche provare come,

NSString *retString = [[result stringByDecodingHTMLEntities] retain]; 
[result release]; 

retString = [retString stripDuplicateNewlineCharacters]; 

Il metodo è definito come,

- (NSString *)stripDuplicateNewlineCharacters { 

    NSMutableString *originalStr = [NSMutableString string]; 

    if (!self) { 
     return nil; 
    } 

    NSArray *componentsArray = [self componentsSeparatedByCharactersInSet:[NSCharacterSet newlineCharacterSet]]; 

    int counter = 0; 
    for (NSString *stringComponent in componentsArray) { 

     counter ++; 

     stringComponent = [stringComponent stringByReplacingOccurrencesOfString:@" " withString:@"<#$%$#>"]; 
     stringComponent = [stringComponent stringByReplacingOccurrencesOfString:@"<#$%$#><#$%$#>" withString:@"<#$%$#>"]; 
     stringComponent = [stringComponent stringByReplacingOccurrencesOfString:@"<#$%$#>" withString:@" "]; 

     if ((stringComponent) && ([stringComponent length] > 0) && (![stringComponent isEqualToString:@" "]) && (![stringComponent isEqualToString:@"\n"])) { 

      if ([componentsArray count] == counter) { 
       [originalStr appendFormat:@"%@", stringComponent]; 
      } else { 
       [originalStr appendFormat:@"%@\n", stringComponent]; 
      } 
     } 
    } 

    return originalStr; 
} 

In questo caso, gli spazi bianchi duplicati vengono rimossi nel metodo stesso durante la rimozione nuova linea personaggi.

Answer 2

Verificare con questo,

//Decode HTML entities and return 
    NSString *retString = [result stringByDecodingHTMLEntities]; 
    [result release]; 

    //Drain 
    [pool drain]; 

    retString = [[retString stringByTrimmingCharactersInSet:[NSCharacterSet whitespaceAndNewlineCharacterSet]] retain]; 

    //Return 
    return [retString autorelease]; 
} 

Se quanto sopra non funziona, provare anche con

completeTrimmed = [completeTrimmed stringByReplacingOccurrencesOfString:@"\n" withString:@""]; 

e

Answer 3

È possibile sostituire @ "/ n/n" con @ "/ n" per ridurre il numero di interruzioni di riga.