Package de.jungblut.crawl
Class SequenceFileResultWriter<T extends FetchResult>
- java.lang.Object
-
- de.jungblut.crawl.SequenceFileResultWriter<T>
-
- All Implemented Interfaces:
ResultWriter<T>,java.lang.AutoCloseable
public class SequenceFileResultWriter<T extends FetchResult> extends java.lang.Object implements ResultWriter<T>
Writes the result into a sequencefile "files/crawl/result.seq". It tab separates the outlinks in the sequencefile value, the key is the origin url.- Author:
- thomas.jungblut
-
-
Field Summary
Fields Modifier and Type Field Description protected org.apache.hadoop.io.SequenceFile.Writerwriter
-
Constructor Summary
Constructors Constructor Description SequenceFileResultWriter()
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description voidclose()org.apache.hadoop.fs.PathgetOutputPath()voidopen(org.apache.hadoop.conf.Configuration conf)Opens the given result writer with a configuration.voidwrite(FetchResult result)Writes a single item to the output.
-
-
-
Method Detail
-
open
public void open(org.apache.hadoop.conf.Configuration conf) throws java.io.IOExceptionDescription copied from interface:ResultWriterOpens the given result writer with a configuration.- Specified by:
openin interfaceResultWriter<T extends FetchResult>- Throws:
java.io.IOException
-
write
public void write(FetchResult result) throws java.io.IOException
Description copied from interface:ResultWriterWrites a single item to the output.- Specified by:
writein interfaceResultWriter<T extends FetchResult>- Throws:
java.io.IOException
-
getOutputPath
public org.apache.hadoop.fs.Path getOutputPath()
-
close
public void close() throws java.lang.Exception- Specified by:
closein interfacejava.lang.AutoCloseable- Throws:
java.lang.Exception
-
-