de.jungblut.crawl.extraction

Class ArticleContentExtrator

java.lang.Object

de.jungblut.crawl.extraction.ArticleContentExtrator

All Implemented Interfaces:

Extractor<ArticleContentExtrator.ContentFetchResult>

public final class ArticleContentExtrator
extends Object
implements Extractor<ArticleContentExtrator.ContentFetchResult>

Extractor for news articles. Uses Boilerpipes ArticleExtractor to extract the largest block of text and the article title.

Author:

thomas.jungblut

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	ArticleContentExtrator.ContentFetchResult Article content fetch result.

Constructor Summary

Constructors

Constructor and Description
ArticleContentExtrator()

Method Summary

Modifier and Type	Method and Description
ArticleContentExtrator.ContentFetchResult	extract(String site) Extracts from a given URL all the content needed and return it.
static String	extractTitle(String html) Extracts the title from the given HTML.
static void	main(String[] args)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ArticleContentExtrator

public ArticleContentExtrator()

Method Detail

extract

public ArticleContentExtrator.ContentFetchResult extract(String site)

Description copied from interface: Extractor

Extracts from a given URL all the content needed and return it. Null if nothing should be returned or could be parsed.

Specified by:

extract in interface Extractor<ArticleContentExtrator.ContentFetchResult>

extractTitle

public static String extractTitle(String html)
                           throws org.htmlparser.util.ParserException

Extracts the title from the given HTML.

Returns:

never null, just an empty string if not parsable.

Throws:

org.htmlparser.util.ParserException

main

public static void main(String[] args)
                 throws IOException,
                        InterruptedException,
                        ExecutionException

Throws:

IOException

InterruptedException

ExecutionException