org.apache.hudi.sink.compact

Class CompactionPlanSourceFunction

java.lang.Object

org.apache.flink.api.common.functions.AbstractRichFunction

org.apache.hudi.sink.compact.CompactionPlanSourceFunction

All Implemented Interfaces:

Serializable, org.apache.flink.api.common.functions.Function, org.apache.flink.api.common.functions.RichFunction, org.apache.flink.streaming.api.functions.source.SourceFunction<CompactionPlanEvent>

public class CompactionPlanSourceFunction
extends org.apache.flink.api.common.functions.AbstractRichFunction
implements org.apache.flink.streaming.api.functions.source.SourceFunction<CompactionPlanEvent>

Flink hudi compaction source function.

This function read the compaction plan as CompactionOperations then assign the compaction task event CompactionPlanEvent to downstream operators.

The compaction instant time is specified explicitly with strategies:

• If the timeline has no inflight instants, use HoodieActiveTimeline.createNewInstantTime() as the instant time;
• If the timeline has inflight instants, use the median instant time between [last complete instant time, earliest inflight instant time] as the instant time.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.flink.streaming.api.functions.source.SourceFunction

org.apache.flink.streaming.api.functions.source.SourceFunction.SourceContext<T>

Field Summary

Fields

Modifier and Type	Field and Description
protected static org.slf4j.Logger	LOG

Constructor Summary

Constructors

Constructor and Description
CompactionPlanSourceFunction(List<Pair<String,HoodieCompactionPlan>> compactionPlans, org.apache.flink.configuration.Configuration conf)

Method Summary

Modifier and Type	Method and Description
void	cancel()
void	close()
void	open(org.apache.flink.configuration.Configuration parameters)
void	run(org.apache.flink.streaming.api.functions.source.SourceFunction.SourceContext sourceContext)

Methods inherited from class org.apache.flink.api.common.functions.AbstractRichFunction

getIterationRuntimeContext, getRuntimeContext, setRuntimeContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

protected static final org.slf4j.Logger LOG

Constructor Detail

CompactionPlanSourceFunction

public CompactionPlanSourceFunction(List<Pair<String,HoodieCompactionPlan>> compactionPlans,
                                    org.apache.flink.configuration.Configuration conf)

Method Detail

open

public void open(org.apache.flink.configuration.Configuration parameters)
          throws Exception

Specified by:

open in interface org.apache.flink.api.common.functions.RichFunction

Overrides:

open in class org.apache.flink.api.common.functions.AbstractRichFunction

Throws:

Exception

run

public void run(org.apache.flink.streaming.api.functions.source.SourceFunction.SourceContext sourceContext)
         throws Exception

Specified by:

run in interface org.apache.flink.streaming.api.functions.source.SourceFunction<CompactionPlanEvent>

Throws:

Exception

close

public void close()
           throws Exception

Specified by:

close in interface org.apache.flink.api.common.functions.RichFunction

Overrides:

close in class org.apache.flink.api.common.functions.AbstractRichFunction

Throws:

Exception

cancel

public void cancel()

Specified by:

cancel in interface org.apache.flink.streaming.api.functions.source.SourceFunction<CompactionPlanEvent>