org.apache.flink.api.java.record.io

Class CsvOutputFormat

java.lang.Object

org.apache.flink.api.common.io.FileOutputFormat<Record>

org.apache.flink.api.java.record.io.FileOutputFormat

org.apache.flink.api.java.record.io.CsvOutputFormat

All Implemented Interfaces:

Serializable, CleanupWhenUnsuccessful, InitializeOnMaster, OutputFormat<Record>

public class CsvOutputFormat
extends FileOutputFormat

This is an OutputFormat to serialize Records to text. The output is structured by record delimiters and field delimiters as common in CSV files. Record delimiter separate records from each other ('\n' is common). Field delimiters separate fields within a record. Record and field delimiters can be configured using the CsvOutputFormat Configuration. The number of fields to serialize must be configured as well. For each field the type of the Value must be specified using the FIELD_TYPE_PARAMETER_PREFIX config key and an index running from 0 to the number of fields. The position within the Record can be configured for each field using the RECORD_POSITION_PARAMETER_PREFIX config key. Either all Record positions must be configured or none. If none is configured, the index of the config key is used.

See Also:

Value, Configuration, Record, Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected static class	CsvOutputFormat.AbstractConfigBuilder<T> Abstract builder used to set parameters to the input format's configuration in a fluent way.
static class	CsvOutputFormat.ConfigBuilder A builder used to set parameters to the input format's configuration in a fluent way.

Nested classes/interfaces inherited from class org.apache.flink.api.common.io.FileOutputFormat

FileOutputFormat.OutputDirectoryMode

Field Summary

Fields

Modifier and Type	Field and Description
static String	FIELD_DELIMITER_PARAMETER
static String	FIELD_TYPE_PARAMETER_PREFIX
static String	LENIENT_PARSING
static String	NUM_FIELDS_PARAMETER
static String	RECORD_DELIMITER_PARAMETER
static String	RECORD_POSITION_PARAMETER_PREFIX

Fields inherited from class org.apache.flink.api.common.io.FileOutputFormat

FILE_PARAMETER_KEY, outputFilePath, stream

Constructor Summary

Constructors

Constructor and Description
CsvOutputFormat()
CsvOutputFormat(Class<? extends Value>... types) Creates an instance of CsvOutputFormat.
CsvOutputFormat(String fieldDelimiter, Class<? extends Value>... types) Creates an instance of CsvOutputFormat.
CsvOutputFormat(String recordDelimiter, String fieldDelimiter, Class<? extends Value>... types) Creates an instance of CsvOutputFormat.

Method Summary

Methods

Modifier and Type	Method and Description
void	close()
void	configure(Configuration parameters)
static CsvOutputFormat.ConfigBuilder	configureRecordFormat(FileDataSink target) Creates a configuration builder that can be used to set the input format's parameters to the config in a fluent fashion.
void	open(int taskNumber, int numTasks)
void	setLenient(boolean lenient)
void	setTypes(Class<? extends Value>... types)
void	writeRecord(Record record)

Methods inherited from class org.apache.flink.api.common.io.FileOutputFormat

configureFileFormat, getDirectoryFileName, getOutputDirectoryMode, getOutputFilePath, getWriteMode, initializeGlobal, setOutputDirectoryMode, setOutputFilePath, setWriteMode, tryCleanupOnError

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

RECORD_DELIMITER_PARAMETER

public static final String RECORD_DELIMITER_PARAMETER

See Also:

Constant Field Values

FIELD_DELIMITER_PARAMETER

public static final String FIELD_DELIMITER_PARAMETER

See Also:

Constant Field Values

NUM_FIELDS_PARAMETER

public static final String NUM_FIELDS_PARAMETER

See Also:

Constant Field Values

FIELD_TYPE_PARAMETER_PREFIX

public static final String FIELD_TYPE_PARAMETER_PREFIX

See Also:

Constant Field Values

RECORD_POSITION_PARAMETER_PREFIX

public static final String RECORD_POSITION_PARAMETER_PREFIX

See Also:

Constant Field Values

LENIENT_PARSING

public static final String LENIENT_PARSING

See Also:

Constant Field Values

Constructor Detail

CsvOutputFormat

public CsvOutputFormat()

CsvOutputFormat

public CsvOutputFormat(Class<? extends Value>... types)

Creates an instance of CsvOutputFormat. The position of the fields in the record is determined by the order in which the classes are given to this constructor. As the default value for separating records '\n' is used. The default field delimiter is ','.

Parameters:

types - The types of the fields that are in the record.

CsvOutputFormat

public CsvOutputFormat(String fieldDelimiter,
               Class<? extends Value>... types)

Creates an instance of CsvOutputFormat. The position of the fields in the record is determined by the order in which the classes are given to this constructor. As the default value for separating records '\n' is used.

Parameters:

fieldDelimiter - The delimiter that is used to separate the different fields in the record.

types - The types of the fields that are in the record.

CsvOutputFormat

public CsvOutputFormat(String recordDelimiter,
               String fieldDelimiter,
               Class<? extends Value>... types)

Creates an instance of CsvOutputFormat. The position of the fields in the record is determined by the order in which the classes are given to this constructor.

Parameters:

recordDelimiter - The delimiter that is used to separate the different records.

fieldDelimiter - The delimiter that is used to separate the different fields in the record.

types - The types of the fields that are in the record.

Method Detail

setTypes

public void setTypes(Class<? extends Value>... types)

setLenient

public void setLenient(boolean lenient)

configure

public void configure(Configuration parameters)

Specified by:

configure in interface OutputFormat<Record>

Overrides:

configure in class FileOutputFormat<Record>

open

public void open(int taskNumber,
        int numTasks)
          throws IOException

Specified by:

open in interface OutputFormat<Record>

Overrides:

open in class FileOutputFormat<Record>

Throws:

IOException

close

public void close()
           throws IOException

Specified by:

close in interface OutputFormat<Record>

Overrides:

close in class FileOutputFormat<Record>

Throws:

IOException

writeRecord

public void writeRecord(Record record)
                 throws IOException

Throws:

IOException

configureRecordFormat

public static CsvOutputFormat.ConfigBuilder configureRecordFormat(FileDataSink target)

Creates a configuration builder that can be used to set the input format's parameters to the config in a fluent fashion.

Returns:

A config builder for setting parameters.