org.apache.hudi.table.action.index

Class AbstractIndexingCatchupTask

java.lang.Object

org.apache.hudi.table.action.index.AbstractIndexingCatchupTask

All Implemented Interfaces:

Runnable, IndexingCatchupTask

Direct Known Subclasses:

RecordBasedIndexingCatchupTask, WriteStatBasedIndexingCatchupTask

public abstract class AbstractIndexingCatchupTask
extends Object
implements IndexingCatchupTask

Indexing check runs for instants that completed after the base instant (in the index plan). It will check if these later instants have logged updates to metadata table or not. If not, then it will do the update. If a later instant is inflight, it will wait until it is completed or the task times out.

Field Summary

Fields

Modifier and Type	Field and Description
protected String	currentCaughtupInstant
protected HoodieEngineContext	engineContext
protected HoodieHeartbeatClient	heartbeatClient
protected List<HoodieInstant>	instantsToIndex
protected HoodieTableMetaClient	metaClient
protected Set<String>	metadataCompletedInstants
protected HoodieTableMetaClient	metadataMetaClient
protected HoodieTableMetadataWriter	metadataWriter
protected HoodieTable	table
protected TransactionManager	transactionManager

Constructor Summary

Constructors

Constructor and Description
AbstractIndexingCatchupTask(HoodieTableMetadataWriter metadataWriter, List<HoodieInstant> instantsToIndex, Set<String> metadataCompletedInstants, HoodieTableMetaClient metaClient, HoodieTableMetaClient metadataMetaClient, TransactionManager transactionManager, String currentCaughtupInstant, HoodieEngineContext engineContext, HoodieTable table, HoodieHeartbeatClient heartbeatClient)

Method Summary

Modifier and Type	Method and Description
void	run()
abstract void	updateIndexForWriteAction(HoodieInstant instant) Updates metadata table for the instant.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

metadataWriter

protected final HoodieTableMetadataWriter metadataWriter

instantsToIndex

protected final List<HoodieInstant> instantsToIndex

metadataCompletedInstants

protected final Set<String> metadataCompletedInstants

metaClient

protected final HoodieTableMetaClient metaClient

metadataMetaClient

protected final HoodieTableMetaClient metadataMetaClient

transactionManager

protected final TransactionManager transactionManager

engineContext

protected final HoodieEngineContext engineContext

table

protected final HoodieTable table

heartbeatClient

protected final HoodieHeartbeatClient heartbeatClient

currentCaughtupInstant

protected String currentCaughtupInstant

Constructor Detail

AbstractIndexingCatchupTask

public AbstractIndexingCatchupTask(HoodieTableMetadataWriter metadataWriter,
                                   List<HoodieInstant> instantsToIndex,
                                   Set<String> metadataCompletedInstants,
                                   HoodieTableMetaClient metaClient,
                                   HoodieTableMetaClient metadataMetaClient,
                                   TransactionManager transactionManager,
                                   String currentCaughtupInstant,
                                   HoodieEngineContext engineContext,
                                   HoodieTable table,
                                   HoodieHeartbeatClient heartbeatClient)

Method Detail

run

public void run()

Specified by:

run in interface Runnable

updateIndexForWriteAction

public abstract void updateIndexForWriteAction(HoodieInstant instant)
                                        throws IOException

Updates metadata table for the instant. This is only called for actions that do actual writes, i.e. for commit/deltacommit/compaction/replacecommit and not for clean/restore/rollback actions.

Specified by:

updateIndexForWriteAction in interface IndexingCatchupTask

Parameters:

instant - HoodieInstant for which to update metadata table

Throws:

IOException