org.apache.hudi.table.action.commit

Class BaseWriteHelper<T extends HoodieRecordPayload,I,K,O,R>

java.lang.Object

org.apache.hudi.table.action.commit.BaseWriteHelper<T,I,K,O,R>

Direct Known Subclasses:

HoodieWriteHelper

public abstract class BaseWriteHelper<T extends HoodieRecordPayload,I,K,O,R>
extends Object

Constructor Summary

Constructors

Constructor and Description
BaseWriteHelper()

Method Summary

Modifier and Type	Method and Description
I	combineOnCondition(boolean condition, I records, int parallelism, HoodieTable<T,I,K,O> table)
abstract I	deduplicateRecords(I records, HoodieIndex<?,?> index, int parallelism)
I	deduplicateRecords(I records, HoodieTable<T,I,K,O> table, int parallelism) Deduplicate Hoodie records, using the given deduplication function.
protected abstract I	tag(I dedupedRecords, HoodieEngineContext context, HoodieTable<T,I,K,O> table)
HoodieWriteMetadata<O>	write(String instantTime, I inputRecords, HoodieEngineContext context, HoodieTable<T,I,K,O> table, boolean shouldCombine, int shuffleParallelism, BaseCommitActionExecutor<T,I,K,O,R> executor, WriteOperationType operationType)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BaseWriteHelper

public BaseWriteHelper()

Method Detail

write

public HoodieWriteMetadata<O> write(String instantTime,
                                    I inputRecords,
                                    HoodieEngineContext context,
                                    HoodieTable<T,I,K,O> table,
                                    boolean shouldCombine,
                                    int shuffleParallelism,
                                    BaseCommitActionExecutor<T,I,K,O,R> executor,
                                    WriteOperationType operationType)

tag

protected abstract I tag(I dedupedRecords,
                         HoodieEngineContext context,
                         HoodieTable<T,I,K,O> table)

combineOnCondition

public I combineOnCondition(boolean condition,
                            I records,
                            int parallelism,
                            HoodieTable<T,I,K,O> table)

deduplicateRecords

public I deduplicateRecords(I records,
                            HoodieTable<T,I,K,O> table,
                            int parallelism)

Deduplicate Hoodie records, using the given deduplication function.

Parameters:

records - hoodieRecords to deduplicate

parallelism - parallelism or partitions to be used while reducing/deduplicating

Returns:

Collection of HoodieRecord already be deduplicated

deduplicateRecords

public abstract I deduplicateRecords(I records,
                                     HoodieIndex<?,?> index,
                                     int parallelism)