org.apache.flink.api.java.hadoop.mapreduce

Class HadoopOutputFormatBase<K,V,T>

java.lang.Object

org.apache.flink.api.java.hadoop.mapreduce.HadoopOutputFormatBase<K,V,T>

All Implemented Interfaces:

Serializable, FinalizeOnMaster, OutputFormat<T>

Direct Known Subclasses:

HadoopOutputFormat

public abstract class HadoopOutputFormatBase<K,V,T>
extends Object
implements OutputFormat<T>, FinalizeOnMaster

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
protected org.apache.hadoop.mapreduce.RecordWriter<K,V>	recordWriter

Constructor Summary

Constructors

Constructor and Description
HadoopOutputFormatBase(org.apache.hadoop.mapreduce.OutputFormat<K,V> mapreduceOutputFormat, org.apache.hadoop.mapreduce.Job job)

Method Summary

Methods

Modifier and Type	Method and Description
void	close() commit the task by moving the output file out from the temporary directory.
void	configure(Configuration parameters)
void	finalizeGlobal(int parallelism)
org.apache.hadoop.conf.Configuration	getConfiguration()
void	open(int taskNumber, int numTasks) create the temporary output file for hadoop RecordWriter.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.api.common.io.OutputFormat

writeRecord

Field Detail

recordWriter

protected transient org.apache.hadoop.mapreduce.RecordWriter<K,V> recordWriter

Constructor Detail

HadoopOutputFormatBase

public HadoopOutputFormatBase(org.apache.hadoop.mapreduce.OutputFormat<K,V> mapreduceOutputFormat,
                      org.apache.hadoop.mapreduce.Job job)

Method Detail

getConfiguration

public org.apache.hadoop.conf.Configuration getConfiguration()

configure

public void configure(Configuration parameters)

Specified by:

configure in interface OutputFormat<T>

open

public void open(int taskNumber,
        int numTasks)
          throws IOException

create the temporary output file for hadoop RecordWriter.

Specified by:

open in interface OutputFormat<T>

Parameters:

taskNumber - The number of the parallel instance.

numTasks - The number of parallel tasks.

Throws:

IOException

close

public void close()
           throws IOException

commit the task by moving the output file out from the temporary directory.

Specified by:

close in interface OutputFormat<T>

Throws:

IOException

finalizeGlobal

public void finalizeGlobal(int parallelism)
                    throws IOException

Specified by:

finalizeGlobal in interface FinalizeOnMaster

Throws:

IOException