de.jungblut.nlp.mr

Class WordCorpusFrequencyJob.DocumentSumReducer

java.lang.Object

org.apache.hadoop.mapreduce.Reducer<org.apache.hadoop.io.Text,TextIntPairWritable,org.apache.hadoop.io.Text,TextIntIntIntWritable>

de.jungblut.nlp.mr.WordCorpusFrequencyJob.DocumentSumReducer

Enclosing class:

WordCorpusFrequencyJob

public static class WordCorpusFrequencyJob.DocumentSumReducer
extends org.apache.hadoop.mapreduce.Reducer<org.apache.hadoop.io.Text,TextIntPairWritable,org.apache.hadoop.io.Text,TextIntIntIntWritable>

Sums up all the documents per token index by docID.
Output is docID as key, value is the token and its document frequency and its term frequency in the doc as well as its index in the dictionary (this must run as single reducer).

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.mapreduce.Reducer

org.apache.hadoop.mapreduce.Reducer.Context

Constructor Summary

Constructors

Constructor and Description
DocumentSumReducer()

Method Summary

Modifier and Type	Method and Description
protected void	cleanup(org.apache.hadoop.mapreduce.Reducer.Context context)
protected void	reduce(org.apache.hadoop.io.Text key, Iterable<TextIntPairWritable> values, org.apache.hadoop.mapreduce.Reducer.Context context)
protected void	setup(org.apache.hadoop.mapreduce.Reducer.Context context)

Methods inherited from class org.apache.hadoop.mapreduce.Reducer

run

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DocumentSumReducer

public DocumentSumReducer()

Method Detail

setup

protected void setup(org.apache.hadoop.mapreduce.Reducer.Context context)
              throws IOException,
                     InterruptedException

Overrides:

setup in class org.apache.hadoop.mapreduce.Reducer<org.apache.hadoop.io.Text,TextIntPairWritable,org.apache.hadoop.io.Text,TextIntIntIntWritable>

Throws:

IOException

InterruptedException

reduce

protected void reduce(org.apache.hadoop.io.Text key,
                      Iterable<TextIntPairWritable> values,
                      org.apache.hadoop.mapreduce.Reducer.Context context)
               throws IOException,
                      InterruptedException

Overrides:

reduce in class org.apache.hadoop.mapreduce.Reducer<org.apache.hadoop.io.Text,TextIntPairWritable,org.apache.hadoop.io.Text,TextIntIntIntWritable>

Throws:

IOException

InterruptedException

cleanup

protected void cleanup(org.apache.hadoop.mapreduce.Reducer.Context context)
                throws IOException,
                       InterruptedException

Overrides:

cleanup in class org.apache.hadoop.mapreduce.Reducer<org.apache.hadoop.io.Text,TextIntPairWritable,org.apache.hadoop.io.Text,TextIntIntIntWritable>

Throws:

IOException

InterruptedException