org.apache.flink.api.java.record.io

Class DelimitedOutputFormat

java.lang.Object

org.apache.flink.api.common.io.FileOutputFormat<org.apache.flink.types.Record>

org.apache.flink.api.java.record.io.FileOutputFormat

org.apache.flink.api.java.record.io.DelimitedOutputFormat

All Implemented Interfaces:

Serializable, org.apache.flink.api.common.io.CleanupWhenUnsuccessful, org.apache.flink.api.common.io.InitializeOnMaster, org.apache.flink.api.common.io.OutputFormat<org.apache.flink.types.Record>

public abstract class DelimitedOutputFormat
extends FileOutputFormat

The base class for output formats that serialize their records into a delimited sequence.

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected static class	DelimitedOutputFormat.AbstractConfigBuilder<T> A builder used to set parameters to the input format's configuration in a fluent way.
static class	DelimitedOutputFormat.ConfigBuilder A builder used to set parameters to the input format's configuration in a fluent way.

Nested classes/interfaces inherited from class org.apache.flink.api.common.io.FileOutputFormat

org.apache.flink.api.common.io.FileOutputFormat.OutputDirectoryMode

Field Summary

Fields

Modifier and Type	Field and Description
static String	RECORD_DELIMITER The configuration key for the entry that defines the record delimiter.
static String	WRITE_BUFFER_SIZE The configuration key for the entry that defines the write-buffer size.

Fields inherited from class org.apache.flink.api.common.io.FileOutputFormat

FILE_PARAMETER_KEY, outputFilePath, stream

Constructor Summary

Constructors

Constructor and Description
DelimitedOutputFormat()

Method Summary

Modifier and Type	Method and Description
void	close()
void	configure(org.apache.flink.configuration.Configuration config) Calls the super classes to configure themselves and reads the config parameters for the delimiter and the write buffer size.
static DelimitedOutputFormat.ConfigBuilder	configureDelimitedFormat(FileDataSink target) Creates a configuration builder that can be used to set the input format's parameters to the config in a fluent fashion.
void	open(int taskNumber, int numTasks)
abstract int	serializeRecord(org.apache.flink.types.Record rec, byte[] target) This method is called for every record so serialize itself into the given target array.
void	writeRecord(org.apache.flink.types.Record record)

Methods inherited from class org.apache.flink.api.common.io.FileOutputFormat

configureFileFormat, getDirectoryFileName, getOutputDirectoryMode, getOutputFilePath, getWriteMode, initializeGlobal, setOutputDirectoryMode, setOutputFilePath, setWriteMode, tryCleanupOnError

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

RECORD_DELIMITER

public static final String RECORD_DELIMITER

The configuration key for the entry that defines the record delimiter.

See Also:

Constant Field Values

WRITE_BUFFER_SIZE

public static final String WRITE_BUFFER_SIZE

The configuration key for the entry that defines the write-buffer size.

See Also:

Constant Field Values

Constructor Detail

DelimitedOutputFormat

public DelimitedOutputFormat()

Method Detail

configure

public void configure(org.apache.flink.configuration.Configuration config)

Calls the super classes to configure themselves and reads the config parameters for the delimiter and the write buffer size.

Specified by:

configure in interface org.apache.flink.api.common.io.OutputFormat<org.apache.flink.types.Record>

Overrides:

configure in class org.apache.flink.api.common.io.FileOutputFormat<org.apache.flink.types.Record>

Parameters:

config - The configuration to read the parameters from.

See Also:

FileOutputFormat.configure(org.apache.flink.configuration.Configuration)

open

public void open(int taskNumber,
                 int numTasks)
          throws IOException

Specified by:

open in interface org.apache.flink.api.common.io.OutputFormat<org.apache.flink.types.Record>

Overrides:

open in class org.apache.flink.api.common.io.FileOutputFormat<org.apache.flink.types.Record>

Throws:

IOException

close

public void close()
           throws IOException

Specified by:

close in interface org.apache.flink.api.common.io.OutputFormat<org.apache.flink.types.Record>

Overrides:

close in class org.apache.flink.api.common.io.FileOutputFormat<org.apache.flink.types.Record>

Throws:

IOException

serializeRecord

public abstract int serializeRecord(org.apache.flink.types.Record rec,
                                    byte[] target)
                             throws Exception

This method is called for every record so serialize itself into the given target array. The method should return the number of bytes occupied in the target array. If the target array is not large enough, a negative value should be returned.

The absolute value of the returned integer can be given as a hint how large an array is required. The array is resized to the return value's absolute value, if that is larger than the current array size. Otherwise, the array size is simply doubled.

Parameters:

rec - The record to be serialized.

target - The array to serialize the record into.

Returns:

The length of the serialized contents, or a negative value, indicating that the array is too small.

Throws:

Exception - If the user code produces an exception that prevents processing the record, it should throw it such that the engine recognizes the situation as a fault.

writeRecord

public void writeRecord(org.apache.flink.types.Record record)
                 throws IOException

Throws:

IOException

configureDelimitedFormat

public static DelimitedOutputFormat.ConfigBuilder configureDelimitedFormat(FileDataSink target)

Creates a configuration builder that can be used to set the input format's parameters to the config in a fluent fashion.

Returns:

A config builder for setting parameters.