org.apache.flink.api.scala.operators

Class ScalaCsvOutputFormat<T extends scala.Product>

java.lang.Object

org.apache.flink.api.common.io.RichOutputFormat<IT>

org.apache.flink.api.common.io.FileOutputFormat<T>

org.apache.flink.api.scala.operators.ScalaCsvOutputFormat<T>

All Implemented Interfaces:

Serializable, CleanupWhenUnsuccessful, InitializeOnMaster, OutputFormat<T>, InputTypeConfigurable

@PublicEvolving
public class ScalaCsvOutputFormat<T extends scala.Product>
extends FileOutputFormat<T>
implements InputTypeConfigurable

This is an OutputFormat to serialize Scala Tuples to text. The output is structured by record delimiters and field delimiters as common in CSV files. Record delimiter separate records from each other ('\n' is common). Field delimiters separate fields within a record.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.flink.api.common.io.FileOutputFormat

FileOutputFormat.OutputDirectoryMode

Field Summary

Fields

Modifier and Type	Field and Description
static String	DEFAULT_FIELD_DELIMITER
static String	DEFAULT_LINE_DELIMITER

Fields inherited from class org.apache.flink.api.common.io.FileOutputFormat

FILE_PARAMETER_KEY, outputFilePath, stream

Constructor Summary

Constructors

Constructor and Description
ScalaCsvOutputFormat(Path outputPath) Creates an instance of CsvOutputFormat.
ScalaCsvOutputFormat(Path outputPath, String fieldDelimiter) Creates an instance of CsvOutputFormat.
ScalaCsvOutputFormat(Path outputPath, String recordDelimiter, String fieldDelimiter) Creates an instance of CsvOutputFormat.

Method Summary

Methods

Modifier and Type	Method and Description
void	close()
void	open(int taskNumber, int numTasks)
void	setAllowNullValues(boolean allowNulls) Configures the format to either allow null values (writing an empty field), or to throw an exception when encountering a null field.
void	setCharsetName(String charsetName) Sets the charset with which the CSV strings are written to the file.
void	setInputType(TypeInformation<?> type, ExecutionConfig executionConfig) The purpose of this method is solely to check whether the data type to be processed is in fact a tuple type.
void	setQuoteStrings(boolean quoteStrings) Configures whether the output format should quote string values.
String	toString()
void	writeRecord(T element)

Methods inherited from class org.apache.flink.api.common.io.FileOutputFormat

configure, getDirectoryFileName, getOutputDirectoryMode, getOutputFilePath, getWriteMode, initializeGlobal, setOutputDirectoryMode, setOutputFilePath, setWriteMode, tryCleanupOnError

Methods inherited from class org.apache.flink.api.common.io.RichOutputFormat

getRuntimeContext, setRuntimeContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

DEFAULT_LINE_DELIMITER

public static final String DEFAULT_LINE_DELIMITER

See Also:

Constant Field Values

DEFAULT_FIELD_DELIMITER

public static final String DEFAULT_FIELD_DELIMITER

Constructor Detail

ScalaCsvOutputFormat

public ScalaCsvOutputFormat(Path outputPath)

Creates an instance of CsvOutputFormat. Lines are separated by the newline character '\n', fields are separated by ','.

Parameters:

outputPath - The path where the CSV file is written.

ScalaCsvOutputFormat

public ScalaCsvOutputFormat(Path outputPath,
                    String fieldDelimiter)

Creates an instance of CsvOutputFormat. Lines are separated by the newline character '\n', fields by the given field delimiter.

Parameters:

outputPath - The path where the CSV file is written.

fieldDelimiter - The delimiter that is used to separate fields in a tuple.

ScalaCsvOutputFormat

public ScalaCsvOutputFormat(Path outputPath,
                    String recordDelimiter,
                    String fieldDelimiter)

Creates an instance of CsvOutputFormat.

Parameters:

outputPath - The path where the CSV file is written.

recordDelimiter - The delimiter that is used to separate the tuples.

fieldDelimiter - The delimiter that is used to separate fields in a tuple.

Method Detail

setAllowNullValues

public void setAllowNullValues(boolean allowNulls)

Configures the format to either allow null values (writing an empty field), or to throw an exception when encountering a null field.

by default, null values are allowed.

Parameters:

allowNulls - Flag to indicate whether the output format should accept null values.

setCharsetName

public void setCharsetName(String charsetName)

Sets the charset with which the CSV strings are written to the file. If not specified, the output format uses the systems default character encoding.

Parameters:

charsetName - The name of charset to use for encoding the output.

setQuoteStrings

public void setQuoteStrings(boolean quoteStrings)

Configures whether the output format should quote string values. String values are fields of type String and StringValue, as well as all subclasses of the latter.

By default, strings are not quoted.

Parameters:

quoteStrings - Flag indicating whether string fields should be quoted.

open

public void open(int taskNumber,
        int numTasks)
          throws IOException

Specified by:

open in interface OutputFormat<T extends scala.Product>

Overrides:

open in class FileOutputFormat<T extends scala.Product>

Throws:

IOException

close

public void close()
           throws IOException

Specified by:

close in interface OutputFormat<T extends scala.Product>

Overrides:

close in class FileOutputFormat<T extends scala.Product>

Throws:

IOException

writeRecord

public void writeRecord(T element)
                 throws IOException

Specified by:

writeRecord in interface OutputFormat<T extends scala.Product>

Throws:

IOException

toString

public String toString()

Overrides:

toString in class Object

setInputType

public void setInputType(TypeInformation<?> type,
                ExecutionConfig executionConfig)

The purpose of this method is solely to check whether the data type to be processed is in fact a tuple type.

Specified by:

setInputType in interface InputTypeConfigurable