org.apache.hudi.sink.bucket

Class ConsistentBucketStreamWriteFunction<I>

java.lang.Object

org.apache.flink.api.common.functions.AbstractRichFunction

org.apache.flink.streaming.api.functions.ProcessFunction<I,Object>

org.apache.hudi.sink.common.AbstractWriteFunction<I>

org.apache.hudi.sink.common.AbstractStreamWriteFunction<I>

org.apache.hudi.sink.StreamWriteFunction<I>

org.apache.hudi.sink.bucket.ConsistentBucketStreamWriteFunction<I>

Type Parameters:

I - the input type

All Implemented Interfaces:

Serializable, org.apache.flink.api.common.functions.Function, org.apache.flink.api.common.functions.RichFunction, org.apache.flink.streaming.api.checkpoint.CheckpointedFunction, org.apache.flink.streaming.api.operators.BoundedOneInput

public class ConsistentBucketStreamWriteFunction<I>
extends StreamWriteFunction<I>

A stream write function with consistent bucket hash index.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hudi.sink.StreamWriteFunction

StreamWriteFunction.DataBucket

Nested classes/interfaces inherited from class org.apache.flink.streaming.api.functions.ProcessFunction

org.apache.flink.streaming.api.functions.ProcessFunction.Context, org.apache.flink.streaming.api.functions.ProcessFunction.OnTimerContext

Field Summary

Fields inherited from class org.apache.hudi.sink.StreamWriteFunction

writeFunction

Fields inherited from class org.apache.hudi.sink.common.AbstractStreamWriteFunction

config, confirming, currentInstant, eventGateway, metaClient, taskID, writeClient, writeStatuses

Constructor Summary

Constructors

Constructor and Description
ConsistentBucketStreamWriteFunction(org.apache.flink.configuration.Configuration config) Constructs a ConsistentBucketStreamWriteFunction.

Method Summary

Modifier and Type	Method and Description
void	open(org.apache.flink.configuration.Configuration parameters)
void	snapshotState()
protected List<WriteStatus>	writeBucket(String instant, StreamWriteFunction.DataBucket bucket, List<HoodieRecord> records)

Methods inherited from class org.apache.hudi.sink.StreamWriteFunction

bufferRecord, close, endInput, getDataBuffer, processElement

Methods inherited from class org.apache.hudi.sink.common.AbstractStreamWriteFunction

handleOperatorEvent, initializeState, instantToWrite, isConfirming, lastPendingInstant, sendBootstrapEvent, setOperatorEventGateway, snapshotState

Methods inherited from class org.apache.flink.streaming.api.functions.ProcessFunction

onTimer

Methods inherited from class org.apache.flink.api.common.functions.AbstractRichFunction

getIterationRuntimeContext, getRuntimeContext, setRuntimeContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ConsistentBucketStreamWriteFunction

public ConsistentBucketStreamWriteFunction(org.apache.flink.configuration.Configuration config)

Constructs a ConsistentBucketStreamWriteFunction.

Parameters:

config - The config options

Method Detail

open

public void open(org.apache.flink.configuration.Configuration parameters)
          throws IOException

Specified by:

open in interface org.apache.flink.api.common.functions.RichFunction

Overrides:

open in class StreamWriteFunction<I>

Throws:

IOException

snapshotState

public void snapshotState()

Overrides:

snapshotState in class StreamWriteFunction<I>

writeBucket

protected List<WriteStatus> writeBucket(String instant,
                                        StreamWriteFunction.DataBucket bucket,
                                        List<HoodieRecord> records)

Overrides:

writeBucket in class StreamWriteFunction<I>