org.apache.flink.streaming.api.functions.sink

Class FileSinkFunction<IN>

java.lang.Object

org.apache.flink.api.common.functions.AbstractRichFunction

org.apache.flink.streaming.api.functions.sink.RichSinkFunction<IN>

org.apache.flink.streaming.api.functions.sink.FileSinkFunction<IN>

Type Parameters:

IN - Input type

All Implemented Interfaces:

Serializable, org.apache.flink.api.common.functions.Function, org.apache.flink.api.common.functions.RichFunction, SinkFunction<IN>

Direct Known Subclasses:

FileSinkFunctionByMillis

public abstract class FileSinkFunction<IN>
extends RichSinkFunction<IN>

Simple implementation of the SinkFunction writing tuples in the specified OutputFormat format. Tuples are collected to a list and written to the file periodically. The target path and the overwrite mode are pre-packaged in format.

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
protected boolean	cleanupCalled
protected int	currentNumberOfSubtasks
protected org.apache.flink.api.common.io.OutputFormat<IN>	format
protected int	indexInSubtaskGroup
protected ArrayList<IN>	tupleList

Constructor Summary

Constructors

Constructor and Description
FileSinkFunction(org.apache.flink.api.common.io.OutputFormat<IN> format)

Method Summary

Modifier and Type	Method and Description
void	close()
protected void	flush()
void	invoke(IN record) Function for standard sink behaviour.
void	open(org.apache.flink.configuration.Configuration parameters)
protected abstract void	resetParameters() Statements to be executed after writing a batch goes here.
protected abstract boolean	updateCondition() Condition for writing the contents of tupleList and clearing it.

Methods inherited from class org.apache.flink.api.common.functions.AbstractRichFunction

getIterationRuntimeContext, getRuntimeContext, setRuntimeContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

tupleList

protected ArrayList<IN> tupleList

format

protected volatile org.apache.flink.api.common.io.OutputFormat<IN> format

cleanupCalled

protected volatile boolean cleanupCalled

indexInSubtaskGroup

protected int indexInSubtaskGroup

currentNumberOfSubtasks

protected int currentNumberOfSubtasks

Constructor Detail

FileSinkFunction

public FileSinkFunction(org.apache.flink.api.common.io.OutputFormat<IN> format)

Method Detail

open

public void open(org.apache.flink.configuration.Configuration parameters)
          throws Exception

Specified by:

open in interface org.apache.flink.api.common.functions.RichFunction

Overrides:

open in class org.apache.flink.api.common.functions.AbstractRichFunction

Throws:

Exception

invoke

public void invoke(IN record)
            throws Exception

Description copied from interface: SinkFunction

Function for standard sink behaviour. This function is called for every record.

Specified by:

invoke in interface SinkFunction<IN>

Specified by:

invoke in class RichSinkFunction<IN>

Parameters:

record - The input record.

Throws:

Exception

close

public void close()
           throws IOException

Specified by:

close in interface org.apache.flink.api.common.functions.RichFunction

Overrides:

close in class org.apache.flink.api.common.functions.AbstractRichFunction

Throws:

IOException

flush

protected void flush()

updateCondition

protected abstract boolean updateCondition()

Condition for writing the contents of tupleList and clearing it.

Returns:

value of the updating condition

resetParameters

protected abstract void resetParameters()

Statements to be executed after writing a batch goes here.