org.apache.hudi.sink

Class CleanFunction<T>

java.lang.Object

org.apache.flink.api.common.functions.AbstractRichFunction

org.apache.hudi.sink.CleanFunction<T>

All Implemented Interfaces:

Serializable, org.apache.flink.api.common.functions.Function, org.apache.flink.api.common.functions.RichFunction, org.apache.flink.api.common.state.CheckpointListener, org.apache.flink.streaming.api.checkpoint.CheckpointedFunction, org.apache.flink.streaming.api.functions.sink.SinkFunction<T>

Direct Known Subclasses:

ClusteringCommitSink, CompactionCommitSink

public class CleanFunction<T>
extends org.apache.flink.api.common.functions.AbstractRichFunction
implements org.apache.flink.streaming.api.functions.sink.SinkFunction<T>, org.apache.flink.streaming.api.checkpoint.CheckpointedFunction, org.apache.flink.api.common.state.CheckpointListener

Sink function that cleans the old commits.

It starts a cleaning task on new checkpoints, there is only one cleaning task at a time, a new task can not be scheduled until the last task finished(fails or normally succeed). The cleaning task never expects to throw but only log.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.flink.streaming.api.functions.sink.SinkFunction

org.apache.flink.streaming.api.functions.sink.SinkFunction.Context

Field Summary

Fields

Modifier and Type	Field and Description
protected HoodieFlinkWriteClient	writeClient

Constructor Summary

Constructors

Constructor and Description
CleanFunction(org.apache.flink.configuration.Configuration conf)

Method Summary

Modifier and Type	Method and Description
void	close()
void	initializeState(org.apache.flink.runtime.state.FunctionInitializationContext context)
void	notifyCheckpointComplete(long l)
void	open(org.apache.flink.configuration.Configuration parameters)
void	snapshotState(org.apache.flink.runtime.state.FunctionSnapshotContext context)

Methods inherited from class org.apache.flink.api.common.functions.AbstractRichFunction

getIterationRuntimeContext, getRuntimeContext, setRuntimeContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.streaming.api.functions.sink.SinkFunction

finish, invoke, invoke, writeWatermark

Methods inherited from interface org.apache.flink.api.common.state.CheckpointListener

notifyCheckpointAborted

Field Detail

writeClient

protected HoodieFlinkWriteClient writeClient

Constructor Detail

CleanFunction

public CleanFunction(org.apache.flink.configuration.Configuration conf)

Method Detail

open

public void open(org.apache.flink.configuration.Configuration parameters)
          throws Exception

Specified by:

open in interface org.apache.flink.api.common.functions.RichFunction

Overrides:

open in class org.apache.flink.api.common.functions.AbstractRichFunction

Throws:

Exception

notifyCheckpointComplete

public void notifyCheckpointComplete(long l)
                              throws Exception

Specified by:

notifyCheckpointComplete in interface org.apache.flink.api.common.state.CheckpointListener

Throws:

Exception

snapshotState

public void snapshotState(org.apache.flink.runtime.state.FunctionSnapshotContext context)
                   throws Exception

Specified by:

snapshotState in interface org.apache.flink.streaming.api.checkpoint.CheckpointedFunction

Throws:

Exception

initializeState

public void initializeState(org.apache.flink.runtime.state.FunctionInitializationContext context)
                     throws Exception

Specified by:

initializeState in interface org.apache.flink.streaming.api.checkpoint.CheckpointedFunction

Throws:

Exception

close

public void close()
           throws Exception

Specified by:

close in interface org.apache.flink.api.common.functions.RichFunction

Overrides:

close in class org.apache.flink.api.common.functions.AbstractRichFunction

Throws:

Exception