org.apache.hudi.table.action.commit

Class BaseCommitActionExecutor<T,I,K,O,R>

java.lang.Object

org.apache.hudi.table.action.BaseActionExecutor<T,I,K,O,R>

org.apache.hudi.table.action.commit.BaseCommitActionExecutor<T,I,K,O,R>

All Implemented Interfaces:

Serializable

public abstract class BaseCommitActionExecutor<T,I,K,O,R>
extends BaseActionExecutor<T,I,K,O,R>

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
protected Option<Map<String,String>>	extraMetadata
protected Option<Pair<HoodieInstant,Map<String,String>>>	lastCompletedTxn
protected WriteOperationType	operationType
protected Set<String>	pendingInflightAndRequestedInstants
protected TaskContextSupplier	taskContextSupplier
protected Option<TransactionManager>	txnManagerOption

Fields inherited from class org.apache.hudi.table.action.BaseActionExecutor

config, context, instantFileNameGenerator, instantFileNameParser, instantGenerator, instantTime, storageConf, table

Constructor Summary

Constructors

Constructor and Description
BaseCommitActionExecutor(HoodieEngineContext context, HoodieWriteConfig config, HoodieTable<T,I,K,O> table, String instantTime, WriteOperationType operationType, Option<Map<String,String>> extraMetadata)

Method Summary

Modifier and Type	Method and Description
protected void	autoCommit(HoodieWriteMetadata<O> result)
protected abstract void	commit(HoodieWriteMetadata<O> result)
protected void	commit(HoodieWriteMetadata<O> result, List<HoodieWriteStat> writeStats)
protected void	commitOnAutoCommit(HoodieWriteMetadata result)
abstract HoodieWriteMetadata<O>	execute(I inputRecords)
protected HoodieWriteMetadata<HoodieData<WriteStatus>>	executeClustering(HoodieClusteringPlan clusteringPlan)
protected void	finalizeWrite(String instantTime, List<HoodieWriteStat> stats, HoodieWriteMetadata<O> result) Finalize Write operation.
protected String	getCommitActionType()
protected String	getSchemaToStoreInCommit() By default, return the writer schema in Write Config for storing in commit.
protected abstract Iterator<List<WriteStatus>>	handleInsert(String idPfx, Iterator<HoodieRecord<T>> recordItr)
protected abstract Iterator<List<WriteStatus>>	handleUpdate(String partitionPath, String fileId, Iterator<HoodieRecord<T>> recordItr)
protected void	runPrecommitValidators(HoodieWriteMetadata<O> writeMetadata) Check if any validators are configured and run those validations.
protected abstract void	setCommitMetadata(HoodieWriteMetadata<O> result)

Methods inherited from class org.apache.hudi.table.action.BaseActionExecutor

dropIndexOnRestore, execute, writeTableMetadata, writeTableMetadata, writeTableMetadata, writeTableMetadata

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

extraMetadata

protected final Option<Map<String,String>> extraMetadata

operationType

protected final WriteOperationType operationType

taskContextSupplier

protected final TaskContextSupplier taskContextSupplier

txnManagerOption

protected final Option<TransactionManager> txnManagerOption

lastCompletedTxn

protected final Option<Pair<HoodieInstant,Map<String,String>>> lastCompletedTxn

pendingInflightAndRequestedInstants

protected final Set<String> pendingInflightAndRequestedInstants

Constructor Detail

BaseCommitActionExecutor

public BaseCommitActionExecutor(HoodieEngineContext context,
                                HoodieWriteConfig config,
                                HoodieTable<T,I,K,O> table,
                                String instantTime,
                                WriteOperationType operationType,
                                Option<Map<String,String>> extraMetadata)

Method Detail

execute

public abstract HoodieWriteMetadata<O> execute(I inputRecords)

getCommitActionType

protected String getCommitActionType()

runPrecommitValidators

protected void runPrecommitValidators(HoodieWriteMetadata<O> writeMetadata)

Check if any validators are configured and run those validations. If any of the validations fail, throws HoodieValidationException.

commitOnAutoCommit

protected void commitOnAutoCommit(HoodieWriteMetadata result)

autoCommit

protected void autoCommit(HoodieWriteMetadata<O> result)

setCommitMetadata

protected abstract void setCommitMetadata(HoodieWriteMetadata<O> result)

commit

protected abstract void commit(HoodieWriteMetadata<O> result)

commit

protected void commit(HoodieWriteMetadata<O> result,
                      List<HoodieWriteStat> writeStats)

finalizeWrite

protected void finalizeWrite(String instantTime,
                             List<HoodieWriteStat> stats,
                             HoodieWriteMetadata<O> result)

Finalize Write operation.

Parameters:

instantTime - Instant Time

stats - Hoodie Write Stat

getSchemaToStoreInCommit

protected String getSchemaToStoreInCommit()

By default, return the writer schema in Write Config for storing in commit.

handleInsert

protected abstract Iterator<List<WriteStatus>> handleInsert(String idPfx,
                                                            Iterator<HoodieRecord<T>> recordItr)
                                                     throws Exception

Throws:

Exception

handleUpdate

protected abstract Iterator<List<WriteStatus>> handleUpdate(String partitionPath,
                                                            String fileId,
                                                            Iterator<HoodieRecord<T>> recordItr)
                                                     throws IOException

Throws:

IOException

executeClustering

protected HoodieWriteMetadata<HoodieData<WriteStatus>> executeClustering(HoodieClusteringPlan clusteringPlan)