org.apache.hudi.table.action.commit

Class HoodieDeleteHelper<T,R>

java.lang.Object

org.apache.hudi.table.action.commit.ParallelismHelper<I>

org.apache.hudi.table.action.commit.BaseDeleteHelper<T,HoodieData<HoodieRecord<T>>,HoodieData<HoodieKey>,HoodieData<WriteStatus>,R>

org.apache.hudi.table.action.commit.HoodieDeleteHelper<T,R>

Type Parameters:

T -

public class HoodieDeleteHelper<T,R>
extends BaseDeleteHelper<T,HoodieData<HoodieRecord<T>>,HoodieData<HoodieKey>,HoodieData<WriteStatus>,R>

A spark implementation of BaseDeleteHelper.

Method Summary

Modifier and Type	Method and Description
static <T> HoodieRecord<T>	createDeleteRecord(HoodieWriteConfig config, HoodieKey key)
static HoodieData	createDeleteRecords(HoodieWriteConfig config, HoodieData<HoodieKey> keys)
HoodieData<HoodieKey>	deduplicateKeys(HoodieData<HoodieKey> keys, HoodieTable<T,HoodieData<HoodieRecord<T>>,HoodieData<HoodieKey>,HoodieData<WriteStatus>> table, int parallelism) Deduplicate Hoodie records, using the given deduplication function.
HoodieWriteMetadata<HoodieData<WriteStatus>>	execute(String instantTime, HoodieData<HoodieKey> keys, HoodieEngineContext context, HoodieWriteConfig config, HoodieTable<T,HoodieData<HoodieRecord<T>>,HoodieData<HoodieKey>,HoodieData<WriteStatus>> table, BaseCommitActionExecutor<T,HoodieData<HoodieRecord<T>>,HoodieData<HoodieKey>,HoodieData<WriteStatus>,R> deleteExecutor)
static HoodieDeleteHelper	newInstance()

Methods inherited from class org.apache.hudi.table.action.commit.ParallelismHelper

deduceShuffleParallelism

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

newInstance

public static HoodieDeleteHelper newInstance()

deduplicateKeys

public HoodieData<HoodieKey> deduplicateKeys(HoodieData<HoodieKey> keys,
                                             HoodieTable<T,HoodieData<HoodieRecord<T>>,HoodieData<HoodieKey>,HoodieData<WriteStatus>> table,
                                             int parallelism)

Description copied from class: BaseDeleteHelper

Deduplicate Hoodie records, using the given deduplication function.

Specified by:

deduplicateKeys in class BaseDeleteHelper<T,HoodieData<HoodieRecord<T>>,HoodieData<HoodieKey>,HoodieData<WriteStatus>,R>

Parameters:

keys - HoodieKeys to deduplicate

table - target Hoodie table for deduplicating

parallelism - parallelism or partitions to be used while reducing/deduplicating

Returns:

HoodieKey already be deduplicated

execute

public HoodieWriteMetadata<HoodieData<WriteStatus>> execute(String instantTime,
                                                            HoodieData<HoodieKey> keys,
                                                            HoodieEngineContext context,
                                                            HoodieWriteConfig config,
                                                            HoodieTable<T,HoodieData<HoodieRecord<T>>,HoodieData<HoodieKey>,HoodieData<WriteStatus>> table,
                                                            BaseCommitActionExecutor<T,HoodieData<HoodieRecord<T>>,HoodieData<HoodieKey>,HoodieData<WriteStatus>,R> deleteExecutor)

Specified by:

execute in class BaseDeleteHelper<T,HoodieData<HoodieRecord<T>>,HoodieData<HoodieKey>,HoodieData<WriteStatus>,R>

createDeleteRecords

public static HoodieData createDeleteRecords(HoodieWriteConfig config,
                                             HoodieData<HoodieKey> keys)

createDeleteRecord

public static <T> HoodieRecord<T> createDeleteRecord(HoodieWriteConfig config,
                                                     HoodieKey key)