org.apache.flink.api.java.record.io

Class DelimitedInputFormat

java.lang.Object

org.apache.flink.api.common.io.FileInputFormat<OT>

org.apache.flink.api.common.io.DelimitedInputFormat<org.apache.flink.types.Record>

org.apache.flink.api.java.record.io.DelimitedInputFormat

All Implemented Interfaces:

Serializable, org.apache.flink.api.common.io.InputFormat<org.apache.flink.types.Record,org.apache.flink.core.fs.FileInputSplit>, org.apache.flink.core.io.InputSplitSource<org.apache.flink.core.fs.FileInputSplit>

Direct Known Subclasses:

TextInputFormat

public abstract class DelimitedInputFormat
extends org.apache.flink.api.common.io.DelimitedInputFormat<org.apache.flink.types.Record>

Base implementation for input formats that split the input at a delimiter into records. The parsing of the record bytes into the record has to be implemented in the readRecord(Record, byte[], int, int) method.

The default delimiter is the newline character '\n'.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.flink.api.common.io.DelimitedInputFormat

org.apache.flink.api.common.io.DelimitedInputFormat.AbstractConfigBuilder<T>, org.apache.flink.api.common.io.DelimitedInputFormat.ConfigBuilder

Nested classes/interfaces inherited from class org.apache.flink.api.common.io.FileInputFormat

org.apache.flink.api.common.io.FileInputFormat.FileBaseStatistics, org.apache.flink.api.common.io.FileInputFormat.InputSplitOpenThread

Field Summary

Fields inherited from class org.apache.flink.api.common.io.DelimitedInputFormat

RECORD_DELIMITER

Fields inherited from class org.apache.flink.api.common.io.FileInputFormat

currentSplit, ENUMERATE_NESTED_FILES_FLAG, enumerateNestedFiles, filePath, INFLATER_INPUT_STREAM_FACTORIES, minSplitSize, numSplits, openTimeout, READ_WHOLE_SPLIT_FLAG, splitLength, splitStart, stream, unsplittable

Constructor Summary

Constructors

Constructor and Description
DelimitedInputFormat()

Method Summary

Modifier and Type	Method and Description
abstract org.apache.flink.types.Record	readRecord(org.apache.flink.types.Record reuse, byte[] bytes, int offset, int numBytes) This function parses the given byte array which represents a serialized key/value pair.

Methods inherited from class org.apache.flink.api.common.io.DelimitedInputFormat

close, configure, configureDelimitedFormat, getBufferSize, getDelimiter, getLineLengthLimit, getNumLineSamples, getStatistics, loadGloablConfigParams, nextRecord, open, reachedEnd, readLine, setBufferSize, setDelimiter, setDelimiter, setDelimiter, setLineLengthLimit, setNumLineSamples

Methods inherited from class org.apache.flink.api.common.io.FileInputFormat

acceptFile, configureFileFormat, createInputSplits, decorateInputStream, extractFileExtension, getFilePath, getFileStats, getInflaterInputStreamFactory, getInputSplitAssigner, getMinSplitSize, getNumSplits, getOpenTimeout, getSplitLength, getSplitStart, registerInflaterInputStreamFactory, setFilePath, setFilePath, setMinSplitSize, setNumSplits, setOpenTimeout, testForUnsplittable, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

DelimitedInputFormat

public DelimitedInputFormat()

Method Detail

readRecord

public abstract org.apache.flink.types.Record readRecord(org.apache.flink.types.Record reuse,
                                                         byte[] bytes,
                                                         int offset,
                                                         int numBytes)

This function parses the given byte array which represents a serialized key/value pair. The parsed content is then returned by setting the pair variables. If the byte array contains invalid content the record can be skipped by returning false.

Specified by:

readRecord in class org.apache.flink.api.common.io.DelimitedInputFormat<org.apache.flink.types.Record>

Parameters:

reuse - The optional reusable holder for the line that is read.

bytes - The serialized record.

Returns:

returns whether the record was successfully deserialized