Class SequenceFileResultWriter<T extends FetchResult>

  • All Implemented Interfaces:
    ResultWriter<T>, java.lang.AutoCloseable

    public class SequenceFileResultWriter<T extends FetchResult>
    extends java.lang.Object
    implements ResultWriter<T>
    Writes the result into a sequencefile "files/crawl/result.seq". It tab separates the outlinks in the sequencefile value, the key is the origin url.
    Author:
    thomas.jungblut
    • Field Summary

      Fields 
      Modifier and Type Field Description
      protected org.apache.hadoop.io.SequenceFile.Writer writer  
    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void close()  
      org.apache.hadoop.fs.Path getOutputPath()  
      void open​(org.apache.hadoop.conf.Configuration conf)
      Opens the given result writer with a configuration.
      void write​(FetchResult result)
      Writes a single item to the output.
      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
    • Field Detail

      • writer

        protected org.apache.hadoop.io.SequenceFile.Writer writer
    • Constructor Detail

      • SequenceFileResultWriter

        public SequenceFileResultWriter()
    • Method Detail

      • open

        public void open​(org.apache.hadoop.conf.Configuration conf)
                  throws java.io.IOException
        Description copied from interface: ResultWriter
        Opens the given result writer with a configuration.
        Specified by:
        open in interface ResultWriter<T extends FetchResult>
        Throws:
        java.io.IOException
      • write

        public void write​(FetchResult result)
                   throws java.io.IOException
        Description copied from interface: ResultWriter
        Writes a single item to the output.
        Specified by:
        write in interface ResultWriter<T extends FetchResult>
        Throws:
        java.io.IOException
      • getOutputPath

        public org.apache.hadoop.fs.Path getOutputPath()
      • close

        public void close()
                   throws java.lang.Exception
        Specified by:
        close in interface java.lang.AutoCloseable
        Throws:
        java.lang.Exception