org.apache.hudi.table.action.commit

Class BaseWriteHelper<T,I,K,O,R>

java.lang.Object

org.apache.hudi.table.action.commit.ParallelismHelper<I>

org.apache.hudi.table.action.commit.BaseWriteHelper<T,I,K,O,R>

Direct Known Subclasses:

HoodieWriteHelper

public abstract class BaseWriteHelper<T,I,K,O,R>
extends ParallelismHelper<I>

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	BaseWriteHelper(SerializableFunctionUnchecked<I,Integer> partitionNumberExtractor)

Method Summary

Modifier and Type	Method and Description
I	combineOnCondition(boolean condition, I records, int configuredParallelism, HoodieTable<T,I,K,O> table)
abstract I	deduplicateRecords(I records, HoodieIndex<?,?> index, int parallelism, String schema, TypedProperties props, HoodieRecordMerger merger)
I	deduplicateRecords(I records, HoodieTable<T,I,K,O> table, int parallelism) Deduplicate Hoodie records, using the given deduplication function.
protected abstract I	tag(I dedupedRecords, HoodieEngineContext context, HoodieTable<T,I,K,O> table)
HoodieWriteMetadata<O>	write(String instantTime, I inputRecords, HoodieEngineContext context, HoodieTable<T,I,K,O> table, boolean shouldCombine, int configuredShuffleParallelism, BaseCommitActionExecutor<T,I,K,O,R> executor, WriteOperationType operationType)

Methods inherited from class org.apache.hudi.table.action.commit.ParallelismHelper

deduceShuffleParallelism

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BaseWriteHelper

protected BaseWriteHelper(SerializableFunctionUnchecked<I,Integer> partitionNumberExtractor)

Method Detail

write

public HoodieWriteMetadata<O> write(String instantTime,
                                    I inputRecords,
                                    HoodieEngineContext context,
                                    HoodieTable<T,I,K,O> table,
                                    boolean shouldCombine,
                                    int configuredShuffleParallelism,
                                    BaseCommitActionExecutor<T,I,K,O,R> executor,
                                    WriteOperationType operationType)

tag

protected abstract I tag(I dedupedRecords,
                         HoodieEngineContext context,
                         HoodieTable<T,I,K,O> table)

combineOnCondition

public I combineOnCondition(boolean condition,
                            I records,
                            int configuredParallelism,
                            HoodieTable<T,I,K,O> table)

deduplicateRecords

public I deduplicateRecords(I records,
                            HoodieTable<T,I,K,O> table,
                            int parallelism)

Deduplicate Hoodie records, using the given deduplication function.

Parameters:

records - hoodieRecords to deduplicate

parallelism - parallelism or partitions to be used while reducing/deduplicating

Returns:

Collection of HoodieRecord already be deduplicated

deduplicateRecords

public abstract I deduplicateRecords(I records,
                                     HoodieIndex<?,?> index,
                                     int parallelism,
                                     String schema,
                                     TypedProperties props,
                                     HoodieRecordMerger merger)