org.apache.flink.api.java.record.io

Class FixedLengthInputFormat

java.lang.Object

org.apache.flink.api.common.io.FileInputFormat<org.apache.flink.types.Record>

org.apache.flink.api.java.record.io.FileInputFormat

org.apache.flink.api.java.record.io.FixedLengthInputFormat

All Implemented Interfaces:

Serializable, org.apache.flink.api.common.io.InputFormat<org.apache.flink.types.Record,org.apache.flink.core.fs.FileInputSplit>, org.apache.flink.core.io.InputSplitSource<org.apache.flink.core.fs.FileInputSplit>

public abstract class FixedLengthInputFormat
extends FileInputFormat

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.flink.api.common.io.FileInputFormat

org.apache.flink.api.common.io.FileInputFormat.AbstractConfigBuilder<T>, org.apache.flink.api.common.io.FileInputFormat.ConfigBuilder, org.apache.flink.api.common.io.FileInputFormat.FileBaseStatistics, org.apache.flink.api.common.io.FileInputFormat.InputSplitOpenThread

Field Summary

Fields

Modifier and Type	Field and Description
static String	RECORDLENGTH_PARAMETER_KEY The config parameter which defines the fixed length of a record.

Fields inherited from class org.apache.flink.api.common.io.FileInputFormat

currentSplit, ENUMERATE_NESTED_FILES_FLAG, enumerateNestedFiles, filePath, INFLATER_INPUT_STREAM_FACTORIES, minSplitSize, numSplits, openTimeout, READ_WHOLE_SPLIT_FLAG, splitLength, splitStart, stream, unsplittable

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	FixedLengthInputFormat() Constructor only sets the key and value classes

Method Summary

Modifier and Type	Method and Description
void	configure(org.apache.flink.configuration.Configuration parameters)
int	getReadBufferSize() Gets the size of the buffer internally used to parse record boundaries.
int	getRecordLength() Returns the fixed length of a record.
org.apache.flink.api.common.io.FileInputFormat.FileBaseStatistics	getStatistics(org.apache.flink.api.common.io.statistics.BaseStatistics cachedStats)
org.apache.flink.types.Record	nextRecord(org.apache.flink.types.Record reuse)
void	open(org.apache.flink.core.fs.FileInputSplit split)
boolean	reachedEnd()
abstract boolean	readBytes(org.apache.flink.types.Record target, byte[] buffer, int startPos) Reads a record out of the given buffer.

Methods inherited from class org.apache.flink.api.common.io.FileInputFormat

acceptFile, close, configureFileFormat, createInputSplits, decorateInputStream, extractFileExtension, getFilePath, getFileStats, getInflaterInputStreamFactory, getInputSplitAssigner, getMinSplitSize, getNumSplits, getOpenTimeout, getSplitLength, getSplitStart, registerInflaterInputStreamFactory, setFilePath, setFilePath, setMinSplitSize, setNumSplits, setOpenTimeout, testForUnsplittable, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

RECORDLENGTH_PARAMETER_KEY

public static final String RECORDLENGTH_PARAMETER_KEY

The config parameter which defines the fixed length of a record.

See Also:

Constant Field Values

Constructor Detail

FixedLengthInputFormat

protected FixedLengthInputFormat()

Constructor only sets the key and value classes

Method Detail

readBytes

public abstract boolean readBytes(org.apache.flink.types.Record target,
                                  byte[] buffer,
                                  int startPos)

Reads a record out of the given buffer. This operation always consumes the standard number of bytes, regardless of whether the produced record was valid.

Parameters:

target - The target Record

buffer - The buffer containing the binary data.

startPos - The start position in the byte array.

Returns:

True, is the record is valid, false otherwise.

getRecordLength

public int getRecordLength()

Returns the fixed length of a record.

Returns:

the fixed length of a record.

getReadBufferSize

public int getReadBufferSize()

Gets the size of the buffer internally used to parse record boundaries.

Returns:

The size of the parsing buffer.

configure

public void configure(org.apache.flink.configuration.Configuration parameters)

Specified by:

configure in interface org.apache.flink.api.common.io.InputFormat<org.apache.flink.types.Record,org.apache.flink.core.fs.FileInputSplit>

Overrides:

configure in class org.apache.flink.api.common.io.FileInputFormat<org.apache.flink.types.Record>

open

public void open(org.apache.flink.core.fs.FileInputSplit split)
          throws IOException

Specified by:

open in interface org.apache.flink.api.common.io.InputFormat<org.apache.flink.types.Record,org.apache.flink.core.fs.FileInputSplit>

Overrides:

open in class org.apache.flink.api.common.io.FileInputFormat<org.apache.flink.types.Record>

Throws:

IOException

getStatistics

public org.apache.flink.api.common.io.FileInputFormat.FileBaseStatistics getStatistics(org.apache.flink.api.common.io.statistics.BaseStatistics cachedStats)
                                                                                throws IOException

Specified by:

getStatistics in interface org.apache.flink.api.common.io.InputFormat<org.apache.flink.types.Record,org.apache.flink.core.fs.FileInputSplit>

Overrides:

getStatistics in class org.apache.flink.api.common.io.FileInputFormat<org.apache.flink.types.Record>

Throws:

IOException

reachedEnd

public boolean reachedEnd()

nextRecord

public org.apache.flink.types.Record nextRecord(org.apache.flink.types.Record reuse)
                                         throws IOException

Throws:

IOException