org.apache.hudi.sink.bucket

Class ConsistentBucketAssignFunction

java.lang.Object

org.apache.flink.api.common.functions.AbstractRichFunction

org.apache.flink.streaming.api.functions.ProcessFunction<HoodieRecord,HoodieRecord>

org.apache.hudi.sink.bucket.ConsistentBucketAssignFunction

All Implemented Interfaces:

Serializable, org.apache.flink.api.common.functions.Function, org.apache.flink.api.common.functions.RichFunction, org.apache.flink.streaming.api.checkpoint.CheckpointedFunction

public class ConsistentBucketAssignFunction
extends org.apache.flink.streaming.api.functions.ProcessFunction<HoodieRecord,HoodieRecord>
implements org.apache.flink.streaming.api.checkpoint.CheckpointedFunction

The function to tag each incoming record with a location of a file based on consistent bucket index.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.flink.streaming.api.functions.ProcessFunction

org.apache.flink.streaming.api.functions.ProcessFunction.Context, org.apache.flink.streaming.api.functions.ProcessFunction.OnTimerContext

Constructor Summary

Constructors

Constructor and Description
ConsistentBucketAssignFunction(org.apache.flink.configuration.Configuration conf)

Method Summary

Modifier and Type	Method and Description
void	initializeState(org.apache.flink.runtime.state.FunctionInitializationContext functionInitializationContext)
void	open(org.apache.flink.configuration.Configuration parameters)
void	processElement(HoodieRecord record, org.apache.flink.streaming.api.functions.ProcessFunction.Context context, org.apache.flink.util.Collector<HoodieRecord> collector)
void	snapshotState(org.apache.flink.runtime.state.FunctionSnapshotContext functionSnapshotContext)

Methods inherited from class org.apache.flink.streaming.api.functions.ProcessFunction

onTimer

Methods inherited from class org.apache.flink.api.common.functions.AbstractRichFunction

close, getIterationRuntimeContext, getRuntimeContext, setRuntimeContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ConsistentBucketAssignFunction

public ConsistentBucketAssignFunction(org.apache.flink.configuration.Configuration conf)

Method Detail

open

public void open(org.apache.flink.configuration.Configuration parameters)
          throws Exception

Specified by:

open in interface org.apache.flink.api.common.functions.RichFunction

Overrides:

open in class org.apache.flink.api.common.functions.AbstractRichFunction

Throws:

Exception

processElement

public void processElement(HoodieRecord record,
                           org.apache.flink.streaming.api.functions.ProcessFunction.Context context,
                           org.apache.flink.util.Collector<HoodieRecord> collector)
                    throws Exception

Specified by:

processElement in class org.apache.flink.streaming.api.functions.ProcessFunction<HoodieRecord,HoodieRecord>

Throws:

Exception

snapshotState

public void snapshotState(org.apache.flink.runtime.state.FunctionSnapshotContext functionSnapshotContext)
                   throws Exception

Specified by:

snapshotState in interface org.apache.flink.streaming.api.checkpoint.CheckpointedFunction

Throws:

Exception

initializeState

public void initializeState(org.apache.flink.runtime.state.FunctionInitializationContext functionInitializationContext)
                     throws Exception

Specified by:

initializeState in interface org.apache.flink.streaming.api.checkpoint.CheckpointedFunction

Throws:

Exception