org.apache.flink.api.java.record.io

Class TextInputFormat

java.lang.Object

org.apache.flink.api.common.io.FileInputFormat<OT>

org.apache.flink.api.common.io.DelimitedInputFormat<org.apache.flink.types.Record>

org.apache.flink.api.java.record.io.DelimitedInputFormat

org.apache.flink.api.java.record.io.TextInputFormat

All Implemented Interfaces:

Serializable, org.apache.flink.api.common.io.InputFormat<org.apache.flink.types.Record,org.apache.flink.core.fs.FileInputSplit>, org.apache.flink.core.io.InputSplitSource<org.apache.flink.core.fs.FileInputSplit>

public class TextInputFormat
extends DelimitedInputFormat

Base implementation for an input format that returns each line as a separate record that contains only a single string, namely the line.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.flink.api.common.io.DelimitedInputFormat

org.apache.flink.api.common.io.DelimitedInputFormat.AbstractConfigBuilder<T>, org.apache.flink.api.common.io.DelimitedInputFormat.ConfigBuilder

Nested classes/interfaces inherited from class org.apache.flink.api.common.io.FileInputFormat

org.apache.flink.api.common.io.FileInputFormat.FileBaseStatistics, org.apache.flink.api.common.io.FileInputFormat.InputSplitOpenThread

Field Summary

Fields

Modifier and Type	Field and Description
protected boolean	ascii
protected ByteBuffer	byteWrapper
static String	CHARSET_NAME
protected CharsetDecoder	decoder
static String	DEFAULT_CHARSET_NAME
static String	FIELD_POS
protected int	pos
protected org.apache.flink.types.StringValue	theString

Fields inherited from class org.apache.flink.api.common.io.DelimitedInputFormat

RECORD_DELIMITER

Fields inherited from class org.apache.flink.api.common.io.FileInputFormat

currentSplit, ENUMERATE_NESTED_FILES_FLAG, enumerateNestedFiles, filePath, INFLATER_INPUT_STREAM_FACTORIES, minSplitSize, numSplits, openTimeout, READ_WHOLE_SPLIT_FLAG, splitLength, splitStart, stream, unsplittable

Constructor Summary

Constructors

Constructor and Description
TextInputFormat()

Method Summary

Modifier and Type	Method and Description
void	configure(org.apache.flink.configuration.Configuration parameters)
org.apache.flink.types.Record	readRecord(org.apache.flink.types.Record reuse, byte[] bytes, int offset, int numBytes) This function parses the given byte array which represents a serialized key/value pair.

Methods inherited from class org.apache.flink.api.common.io.DelimitedInputFormat

close, configureDelimitedFormat, getBufferSize, getDelimiter, getLineLengthLimit, getNumLineSamples, getStatistics, loadGloablConfigParams, nextRecord, open, reachedEnd, readLine, setBufferSize, setDelimiter, setDelimiter, setDelimiter, setLineLengthLimit, setNumLineSamples

Methods inherited from class org.apache.flink.api.common.io.FileInputFormat

acceptFile, configureFileFormat, createInputSplits, decorateInputStream, extractFileExtension, getFilePath, getFileStats, getInflaterInputStreamFactory, getInputSplitAssigner, getMinSplitSize, getNumSplits, getOpenTimeout, getSplitLength, getSplitStart, registerInflaterInputStreamFactory, setFilePath, setFilePath, setMinSplitSize, setNumSplits, setOpenTimeout, testForUnsplittable, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

CHARSET_NAME

public static final String CHARSET_NAME

See Also:

Constant Field Values

FIELD_POS

public static final String FIELD_POS

See Also:

Constant Field Values

DEFAULT_CHARSET_NAME

public static final String DEFAULT_CHARSET_NAME

See Also:

Constant Field Values

theString

protected final org.apache.flink.types.StringValue theString

decoder

protected transient CharsetDecoder decoder

byteWrapper

protected transient ByteBuffer byteWrapper

pos

protected transient int pos

ascii

protected transient boolean ascii

Constructor Detail

TextInputFormat

public TextInputFormat()

Method Detail

configure

public void configure(org.apache.flink.configuration.Configuration parameters)

Specified by:

configure in interface org.apache.flink.api.common.io.InputFormat<org.apache.flink.types.Record,org.apache.flink.core.fs.FileInputSplit>

Overrides:

configure in class org.apache.flink.api.common.io.DelimitedInputFormat<org.apache.flink.types.Record>

readRecord

public org.apache.flink.types.Record readRecord(org.apache.flink.types.Record reuse,
                                                byte[] bytes,
                                                int offset,
                                                int numBytes)

Description copied from class: DelimitedInputFormat

This function parses the given byte array which represents a serialized key/value pair. The parsed content is then returned by setting the pair variables. If the byte array contains invalid content the record can be skipped by returning false.

Specified by:

readRecord in class DelimitedInputFormat

Parameters:

reuse - The optional reusable holder for the line that is read.

bytes - The serialized record.

Returns:

returns whether the record was successfully deserialized