de.jungblut.crawl

Class SequentialCrawler<T extends FetchResult>

java.lang.Object

de.jungblut.crawl.SequentialCrawler<T>

All Implemented Interfaces:

Crawler<T>

public final class SequentialCrawler<T extends FetchResult>
extends Object
implements Crawler<T>

Sequential crawler, mainly for debugging or development.

Author:

thomas.jungblut

Constructor Summary

Constructors

Constructor and Description
SequentialCrawler(int fetches, Extractor<T> extractor, ResultWriter<T> writer)

Method Summary

Modifier and Type	Method and Description
void	process(String... seedUrl) Starts the crawler, starting by the seedURL.
void	setup(int fetches, Extractor<T> extractor, ResultWriter<T> writer) Setups this crawler.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SequentialCrawler

public SequentialCrawler(int fetches,
                         Extractor<T> extractor,
                         ResultWriter<T> writer)
                  throws IOException

Throws:

IOException

Method Detail

setup

public final void setup(int fetches,
                        Extractor<T> extractor,
                        ResultWriter<T> writer)
                 throws IOException

Description copied from interface: Crawler

Setups this crawler.

Specified by:

setup in interface Crawler<T extends FetchResult>

Parameters:

fetches - how many maximum fetches it should do.

extractor - the given Extractor to extract a FetchResult.

writer - the ResultWriter to write the result to a sink.

Throws:

IOException

process

public final void process(String... seedUrl)
                   throws InterruptedException,
                          ExecutionException

Description copied from interface: Crawler

Starts the crawler, starting by the seedURL. The real logic is implemented by the crawler itself.

Specified by:

process in interface Crawler<T extends FetchResult>

Throws:

InterruptedException

ExecutionException