org.apache.flink.formats.hadoop.bulk

Class DefaultHadoopFileCommitterFactory

java.lang.Object

org.apache.flink.formats.hadoop.bulk.DefaultHadoopFileCommitterFactory

All Implemented Interfaces:

Serializable, HadoopFileCommitterFactory

public class DefaultHadoopFileCommitterFactory
extends Object
implements HadoopFileCommitterFactory

The default hadoop file committer factory which always use HadoopRenameFileCommitter.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
DefaultHadoopFileCommitterFactory()

Method Summary

Modifier and Type	Method and Description
HadoopFileCommitter	create(org.apache.hadoop.conf.Configuration configuration, org.apache.hadoop.fs.Path targetFilePath) Creates a new Hadoop file committer for writing.
HadoopFileCommitter	recoverForCommit(org.apache.hadoop.conf.Configuration configuration, org.apache.hadoop.fs.Path targetFilePath, org.apache.hadoop.fs.Path tempFilePath) Creates a Hadoop file committer for commit the pending file.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DefaultHadoopFileCommitterFactory

public DefaultHadoopFileCommitterFactory()

Method Detail

create

public HadoopFileCommitter create(org.apache.hadoop.conf.Configuration configuration,
                                  org.apache.hadoop.fs.Path targetFilePath)
                           throws IOException

Description copied from interface: HadoopFileCommitterFactory

Creates a new Hadoop file committer for writing.

Specified by:

create in interface HadoopFileCommitterFactory

Parameters:

configuration - The hadoop configuration.

targetFilePath - The target path to commit to.

Returns:

The corresponding Hadoop file committer.

Throws:

IOException

recoverForCommit

public HadoopFileCommitter recoverForCommit(org.apache.hadoop.conf.Configuration configuration,
                                            org.apache.hadoop.fs.Path targetFilePath,
                                            org.apache.hadoop.fs.Path tempFilePath)
                                     throws IOException

Description copied from interface: HadoopFileCommitterFactory

Creates a Hadoop file committer for commit the pending file.

Specified by:

recoverForCommit in interface HadoopFileCommitterFactory

Parameters:

configuration - The hadoop configuration.

targetFilePath - The target path to commit to.

tempFilePath - The path of the remaining pending file.

Returns:

The corresponding Hadoop file committer.

Throws:

IOException