| Modifier and Type | Method and Description |
|---|---|
protected HoodieTable |
BaseHoodieWriteClient.createTable(HoodieWriteConfig config,
org.apache.hadoop.conf.Configuration hadoopConf) |
protected abstract HoodieTable |
BaseHoodieWriteClient.createTable(HoodieWriteConfig config,
org.apache.hadoop.conf.Configuration hadoopConf,
boolean refreshTimeline) |
protected abstract HoodieTable |
BaseHoodieWriteClient.doInitTable(HoodieTableMetaClient metaClient,
Option<String> instantTime,
boolean initialMetadataTableIfNecessary)
Instantiates engine-specific instance of
HoodieTable as well as performs necessary
bootstrapping operations (for ex, validating whether Metadata Table has to be bootstrapped)
NOTE: THIS OPERATION IS EXECUTED UNDER LOCK, THEREFORE SHOULD AVOID ANY OPERATIONS
NOT REQUIRING EXTERNAL SYNCHRONIZATION |
protected HoodieTable |
BaseHoodieWriteClient.initTable(WriteOperationType operationType,
Option<String> instantTime) |
protected HoodieTable |
BaseHoodieWriteClient.initTable(WriteOperationType operationType,
Option<String> instantTime,
boolean initialMetadataTableIfNecessary)
Instantiates and initializes instance of
HoodieTable, performing crucial bootstrapping
operations such as:
NOTE: This method is engine-agnostic and SHOULD NOT be overloaded, please check on
BaseHoodieWriteClient.doInitTable(HoodieTableMetaClient, Option, boolean) instead
Checking whether upgrade/downgrade is required
Bootstrapping Metadata Table (if required)
Initializing metrics contexts
|
| Modifier and Type | Method and Description |
|---|---|
protected void |
BaseHoodieWriteClient.archive(HoodieTable table,
boolean acquireLockForArchival)
Trigger archival for the table.
|
protected void |
BaseHoodieWriteClient.autoArchiveOnCommit(HoodieTable table,
boolean acquireLockForArchival) |
protected void |
BaseHoodieWriteClient.commit(HoodieTable table,
String commitActionType,
String instantTime,
HoodieCommitMetadata metadata,
List<HoodieWriteStat> stats) |
protected abstract void |
BaseHoodieWriteClient.completeCompaction(HoodieCommitMetadata metadata,
HoodieTable table,
String compactionCommitTime)
Commit Compaction and track metrics.
|
protected void |
BaseHoodieWriteClient.finalizeWrite(HoodieTable table,
String instantTime,
List<HoodieWriteStat> stats)
Finalize Write operation.
|
protected void |
BaseHoodieWriteClient.postCommit(HoodieTable table,
HoodieCommitMetadata metadata,
String instantTime,
Option<Map<String,String>> extraMetadata,
boolean acquireLockForArchival)
Post Commit Hook.
|
protected abstract O |
BaseHoodieWriteClient.postWrite(HoodieWriteMetadata<O> result,
String instantTime,
HoodieTable hoodieTable)
Common method containing steps to be performed after write (upsert/insert/..) operations including auto-commit.
|
protected void |
BaseHoodieWriteClient.rollbackInflightClustering(HoodieInstant inflightInstant,
HoodieTable table) |
protected void |
BaseHoodieWriteClient.runAnyPendingClustering(HoodieTable table) |
protected void |
BaseHoodieWriteClient.runAnyPendingCompactions(HoodieTable table) |
protected void |
BaseHoodieWriteClient.runTableServicesInline(HoodieTable table,
HoodieCommitMetadata metadata,
Option<Map<String,String>> extraMetadata) |
protected void |
BaseHoodieWriteClient.writeTableMetadata(HoodieTable table,
String instantTime,
String actionType,
HoodieCommitMetadata metadata)
Write the HoodieCommitMetadata to metadata table if available.
|
| Constructor and Description |
|---|
HoodieTimelineArchiver(HoodieWriteConfig config,
HoodieTable<T,I,K,O> table) |
| Modifier and Type | Method and Description |
|---|---|
static void |
HeartbeatUtils.abortIfHeartbeatExpired(String instantTime,
HoodieTable table,
HoodieHeartbeatClient heartbeatClient,
HoodieWriteConfig config)
Check if the heartbeat corresponding to instantTime has expired.
|
| Modifier and Type | Method and Description |
|---|---|
Option<HoodieCommitMetadata> |
SimpleConcurrentFileWritesConflictResolutionStrategy.resolveConflict(HoodieTable table,
ConcurrentOperation thisOperation,
ConcurrentOperation otherOperation) |
Option<HoodieCommitMetadata> |
ConflictResolutionStrategy.resolveConflict(HoodieTable table,
ConcurrentOperation thisOperation,
ConcurrentOperation otherOperation)
Implementations of this method will determine how to resolve a conflict between 2 commits.
|
| Modifier and Type | Method and Description |
|---|---|
static Option<HoodieCommitMetadata> |
TransactionUtils.resolveWriteConflictIfAny(HoodieTable table,
Option<HoodieInstant> currentTxnOwnerInstant,
Option<HoodieCommitMetadata> thisCommitMetadata,
HoodieWriteConfig config,
Option<HoodieInstant> lastCompletedTxnOwnerInstant,
boolean reloadActiveTimeline,
Set<String> pendingInstants)
Resolve any write conflicts when committing data.
|
| Modifier and Type | Field and Description |
|---|---|
protected HoodieTable |
HoodieLazyInsertIterable.hoodieTable |
| Constructor and Description |
|---|
CopyOnWriteInsertHandler(HoodieWriteConfig config,
String instantTime,
boolean areRecordsSorted,
HoodieTable hoodieTable,
String idPrefix,
TaskContextSupplier taskContextSupplier,
WriteHandleFactory writeHandleFactory) |
HoodieLazyInsertIterable(Iterator<HoodieRecord<T>> recordItr,
boolean areRecordsSorted,
HoodieWriteConfig config,
String instantTime,
HoodieTable hoodieTable,
String idPrefix,
TaskContextSupplier taskContextSupplier) |
HoodieLazyInsertIterable(Iterator<HoodieRecord<T>> recordItr,
boolean areRecordsSorted,
HoodieWriteConfig config,
String instantTime,
HoodieTable hoodieTable,
String idPrefix,
TaskContextSupplier taskContextSupplier,
WriteHandleFactory writeHandleFactory) |
| Modifier and Type | Method and Description |
|---|---|
static List<Pair<String,HoodieBaseFile>> |
HoodieIndexUtils.getLatestBaseFilesForAllPartitions(List<String> partitions,
HoodieEngineContext context,
HoodieTable hoodieTable)
Fetches Pair of partition path and
HoodieBaseFiles for interested partitions. |
static List<HoodieBaseFile> |
HoodieIndexUtils.getLatestBaseFilesForPartition(String partition,
HoodieTable hoodieTable)
Fetches Pair of partition path and
HoodieBaseFiles for interested partitions. |
abstract <R> HoodieData<HoodieRecord<R>> |
HoodieIndex.tagLocation(HoodieData<HoodieRecord<R>> records,
HoodieEngineContext context,
HoodieTable hoodieTable)
Looks up the index and tags each incoming record with a location of a file that contains
the row (if it is actually present).
|
I |
HoodieIndex.tagLocation(I records,
HoodieEngineContext context,
HoodieTable hoodieTable)
Deprecated.
|
abstract HoodieData<WriteStatus> |
HoodieIndex.updateLocation(HoodieData<WriteStatus> writeStatuses,
HoodieEngineContext context,
HoodieTable hoodieTable)
Extracts the location of written records, and updates the index.
|
O |
HoodieIndex.updateLocation(O writeStatuses,
HoodieEngineContext context,
HoodieTable hoodieTable)
Deprecated.
|
| Modifier and Type | Method and Description |
|---|---|
HoodiePairData<HoodieKey,HoodieRecordLocation> |
ListBasedHoodieBloomIndexHelper.findMatchingFilesForRecordKeys(HoodieWriteConfig config,
HoodieEngineContext context,
HoodieTable hoodieTable,
HoodiePairData<String,String> partitionRecordKeyPairs,
HoodieData<Pair<String,HoodieKey>> fileComparisonPairs,
Map<String,List<BloomIndexFileInfo>> partitionToFileInfo,
Map<String,Long> recordsPerPartition) |
abstract HoodiePairData<HoodieKey,HoodieRecordLocation> |
BaseHoodieBloomIndexHelper.findMatchingFilesForRecordKeys(HoodieWriteConfig config,
HoodieEngineContext context,
HoodieTable hoodieTable,
HoodiePairData<String,String> partitionRecordKeyPairs,
HoodieData<Pair<String,HoodieKey>> fileComparisonPairs,
Map<String,List<BloomIndexFileInfo>> partitionToFileInfo,
Map<String,Long> recordsPerPartition)
Find out
|
protected List<Pair<String,BloomIndexFileInfo>> |
HoodieBloomIndex.loadColumnRangesFromMetaIndex(List<String> partitions,
HoodieEngineContext context,
HoodieTable<?,?,?,?> hoodieTable)
Load the column stats index as BloomIndexFileInfo for all the involved files in the partition.
|
<R> HoodieData<HoodieRecord<R>> |
HoodieBloomIndex.tagLocation(HoodieData<HoodieRecord<R>> records,
HoodieEngineContext context,
HoodieTable hoodieTable) |
HoodieData<WriteStatus> |
HoodieBloomIndex.updateLocation(HoodieData<WriteStatus> writeStatusData,
HoodieEngineContext context,
HoodieTable hoodieTable) |
| Constructor and Description |
|---|
HoodieBaseBloomIndexCheckFunction(HoodieTable hoodieTable,
HoodieWriteConfig config) |
| Modifier and Type | Method and Description |
|---|---|
<R> HoodieData<HoodieRecord<R>> |
HoodieBucketIndex.tagLocation(HoodieData<HoodieRecord<R>> records,
HoodieEngineContext context,
HoodieTable hoodieTable) |
HoodieData<WriteStatus> |
HoodieBucketIndex.updateLocation(HoodieData<WriteStatus> writeStatuses,
HoodieEngineContext context,
HoodieTable hoodieTable) |
| Modifier and Type | Method and Description |
|---|---|
<R> HoodieData<HoodieRecord<R>> |
HoodieInMemoryHashIndex.tagLocation(HoodieData<HoodieRecord<R>> records,
HoodieEngineContext context,
HoodieTable hoodieTable) |
HoodieData<WriteStatus> |
HoodieInMemoryHashIndex.updateLocation(HoodieData<WriteStatus> writeStatuses,
HoodieEngineContext context,
HoodieTable hoodieTable) |
| Modifier and Type | Method and Description |
|---|---|
protected HoodiePairData<HoodieKey,HoodieRecordLocation> |
HoodieGlobalSimpleIndex.fetchAllRecordLocations(HoodieEngineContext context,
HoodieTable hoodieTable,
int parallelism)
Fetch record locations for passed in
HoodieKeys. |
protected HoodiePairData<HoodieKey,HoodieRecordLocation> |
HoodieSimpleIndex.fetchRecordLocations(HoodieEngineContext context,
HoodieTable hoodieTable,
int parallelism,
List<Pair<String,HoodieBaseFile>> baseFiles) |
protected HoodiePairData<HoodieKey,HoodieRecordLocation> |
HoodieSimpleIndex.fetchRecordLocationsForAffectedPartitions(HoodieData<HoodieKey> hoodieKeys,
HoodieEngineContext context,
HoodieTable hoodieTable,
int parallelism)
Fetch record locations for passed in
HoodieKeys. |
protected List<Pair<String,HoodieBaseFile>> |
HoodieGlobalSimpleIndex.getAllBaseFilesInTable(HoodieEngineContext context,
HoodieTable hoodieTable)
Load all files for all partitions as
|
<R> HoodieData<HoodieRecord<R>> |
HoodieGlobalSimpleIndex.tagLocation(HoodieData<HoodieRecord<R>> records,
HoodieEngineContext context,
HoodieTable hoodieTable) |
<R> HoodieData<HoodieRecord<R>> |
HoodieSimpleIndex.tagLocation(HoodieData<HoodieRecord<R>> records,
HoodieEngineContext context,
HoodieTable hoodieTable) |
protected <R> HoodieData<HoodieRecord<R>> |
HoodieGlobalSimpleIndex.tagLocationInternal(HoodieData<HoodieRecord<R>> inputRecords,
HoodieEngineContext context,
HoodieTable hoodieTable)
Tags records location for incoming records.
|
protected <R> HoodieData<HoodieRecord<R>> |
HoodieSimpleIndex.tagLocationInternal(HoodieData<HoodieRecord<R>> inputRecords,
HoodieEngineContext context,
HoodieTable hoodieTable)
Tags records location for incoming records.
|
HoodieData<WriteStatus> |
HoodieSimpleIndex.updateLocation(HoodieData<WriteStatus> writeStatuses,
HoodieEngineContext context,
HoodieTable hoodieTable) |
| Modifier and Type | Field and Description |
|---|---|
protected HoodieTable<T,I,K,O> |
HoodieIOHandle.hoodieTable |
| Modifier and Type | Method and Description |
|---|---|
abstract HoodieWriteHandle<T,I,K,O> |
WriteHandleFactory.create(HoodieWriteConfig config,
String commitTime,
HoodieTable<T,I,K,O> hoodieTable,
String partitionPath,
String fileIdPrefix,
TaskContextSupplier taskContextSupplier) |
HoodieAppendHandle<T,I,K,O> |
AppendHandleFactory.create(HoodieWriteConfig hoodieConfig,
String commitTime,
HoodieTable<T,I,K,O> hoodieTable,
String partitionPath,
String fileIdPrefix,
TaskContextSupplier sparkTaskContextSupplier) |
HoodieWriteHandle<T,I,K,O> |
CreateHandleFactory.create(HoodieWriteConfig hoodieConfig,
String commitTime,
HoodieTable<T,I,K,O> hoodieTable,
String partitionPath,
String fileIdPrefix,
TaskContextSupplier taskContextSupplier) |
HoodieWriteHandle<T,I,K,O> |
SingleFileHandleCreateFactory.create(HoodieWriteConfig hoodieConfig,
String commitTime,
HoodieTable<T,I,K,O> hoodieTable,
String partitionPath,
String fileIdPrefix,
TaskContextSupplier taskContextSupplier) |
protected HoodieFileWriter |
HoodieWriteHandle.createNewFileWriter(String instantTime,
org.apache.hadoop.fs.Path path,
HoodieTable<T,I,K,O> hoodieTable,
HoodieWriteConfig config,
org.apache.avro.Schema schema,
TaskContextSupplier taskContextSupplier) |
| Constructor and Description |
|---|
HoodieAppendHandle(HoodieWriteConfig config,
String instantTime,
HoodieTable<T,I,K,O> hoodieTable,
String partitionPath,
String fileId,
Iterator<HoodieRecord<T>> recordItr,
TaskContextSupplier taskContextSupplier) |
HoodieAppendHandle(HoodieWriteConfig config,
String instantTime,
HoodieTable<T,I,K,O> hoodieTable,
String partitionPath,
String fileId,
TaskContextSupplier sparkTaskContextSupplier) |
HoodieBootstrapHandle(HoodieWriteConfig config,
String commitTime,
HoodieTable<T,I,K,O> hoodieTable,
String partitionPath,
String fileId,
TaskContextSupplier taskContextSupplier) |
HoodieConcatHandle(HoodieWriteConfig config,
String instantTime,
HoodieTable<T,I,K,O> hoodieTable,
Iterator<HoodieRecord<T>> recordItr,
String partitionPath,
String fileId,
TaskContextSupplier taskContextSupplier,
Option<BaseKeyGenerator> keyGeneratorOpt) |
HoodieConcatHandle(HoodieWriteConfig config,
String instantTime,
HoodieTable hoodieTable,
Map<String,HoodieRecord<T>> keyToNewRecords,
String partitionPath,
String fileId,
HoodieBaseFile dataFileToBeMerged,
TaskContextSupplier taskContextSupplier) |
HoodieCreateHandle(HoodieWriteConfig config,
String instantTime,
HoodieTable<T,I,K,O> hoodieTable,
String partitionPath,
String fileId,
Map<String,HoodieRecord<T>> recordMap,
TaskContextSupplier taskContextSupplier)
Called by the compactor code path.
|
HoodieCreateHandle(HoodieWriteConfig config,
String instantTime,
HoodieTable<T,I,K,O> hoodieTable,
String partitionPath,
String fileId,
Option<org.apache.avro.Schema> overriddenSchema,
TaskContextSupplier taskContextSupplier) |
HoodieCreateHandle(HoodieWriteConfig config,
String instantTime,
HoodieTable<T,I,K,O> hoodieTable,
String partitionPath,
String fileId,
Option<org.apache.avro.Schema> overriddenSchema,
TaskContextSupplier taskContextSupplier,
boolean preserveMetadata) |
HoodieCreateHandle(HoodieWriteConfig config,
String instantTime,
HoodieTable<T,I,K,O> hoodieTable,
String partitionPath,
String fileId,
TaskContextSupplier taskContextSupplier) |
HoodieCreateHandle(HoodieWriteConfig config,
String instantTime,
HoodieTable<T,I,K,O> hoodieTable,
String partitionPath,
String fileId,
TaskContextSupplier taskContextSupplier,
boolean preserveMetadata) |
HoodieKeyLocationFetchHandle(HoodieWriteConfig config,
HoodieTable<T,I,K,O> hoodieTable,
Pair<String,HoodieBaseFile> partitionPathBaseFilePair,
Option<BaseKeyGenerator> keyGeneratorOpt) |
HoodieKeyLookupHandle(HoodieWriteConfig config,
HoodieTable<T,I,K,O> hoodieTable,
Pair<String,String> partitionPathFileIDPair) |
HoodieMergeHandle(HoodieWriteConfig config,
String instantTime,
HoodieTable<T,I,K,O> hoodieTable,
Iterator<HoodieRecord<T>> recordItr,
String partitionPath,
String fileId,
TaskContextSupplier taskContextSupplier,
HoodieBaseFile baseFile,
Option<BaseKeyGenerator> keyGeneratorOpt) |
HoodieMergeHandle(HoodieWriteConfig config,
String instantTime,
HoodieTable<T,I,K,O> hoodieTable,
Iterator<HoodieRecord<T>> recordItr,
String partitionPath,
String fileId,
TaskContextSupplier taskContextSupplier,
Option<BaseKeyGenerator> keyGeneratorOpt) |
HoodieMergeHandle(HoodieWriteConfig config,
String instantTime,
HoodieTable<T,I,K,O> hoodieTable,
Map<String,HoodieRecord<T>> keyToNewRecords,
String partitionPath,
String fileId,
HoodieBaseFile dataFileToBeMerged,
TaskContextSupplier taskContextSupplier,
Option<BaseKeyGenerator> keyGeneratorOpt)
Called by compactor code path.
|
HoodieRangeInfoHandle(HoodieWriteConfig config,
HoodieTable<T,I,K,O> hoodieTable,
Pair<String,String> partitionPathFilePair) |
HoodieReadHandle(HoodieWriteConfig config,
HoodieTable<T,I,K,O> hoodieTable,
Pair<String,String> partitionPathFileIDPair) |
HoodieSortedMergeHandle(HoodieWriteConfig config,
String instantTime,
HoodieTable<T,I,K,O> hoodieTable,
Iterator<HoodieRecord<T>> recordItr,
String partitionPath,
String fileId,
TaskContextSupplier taskContextSupplier,
Option<BaseKeyGenerator> keyGeneratorOpt) |
HoodieSortedMergeHandle(HoodieWriteConfig config,
String instantTime,
HoodieTable<T,I,K,O> hoodieTable,
Map<String,HoodieRecord<T>> keyToNewRecordsOrig,
String partitionPath,
String fileId,
HoodieBaseFile dataFileToBeMerged,
TaskContextSupplier taskContextSupplier,
Option<BaseKeyGenerator> keyGeneratorOpt)
Called by compactor code path.
|
HoodieUnboundedCreateHandle(HoodieWriteConfig config,
String instantTime,
HoodieTable<T,I,K,O> hoodieTable,
String partitionPath,
String fileId,
TaskContextSupplier taskContextSupplier,
boolean preserveHoodieMetadata) |
HoodieWriteHandle(HoodieWriteConfig config,
String instantTime,
String partitionPath,
String fileId,
HoodieTable<T,I,K,O> hoodieTable,
Option<org.apache.avro.Schema> overriddenSchema,
TaskContextSupplier taskContextSupplier) |
HoodieWriteHandle(HoodieWriteConfig config,
String instantTime,
String partitionPath,
String fileId,
HoodieTable<T,I,K,O> hoodieTable,
TaskContextSupplier taskContextSupplier) |
| Modifier and Type | Method and Description |
|---|---|
static <T extends HoodieRecordPayload,R extends org.apache.avro.generic.IndexedRecord,I,K,O> |
HoodieFileWriterFactory.getFileWriter(String instantTime,
org.apache.hadoop.fs.Path path,
HoodieTable<T,I,K,O> hoodieTable,
HoodieWriteConfig config,
org.apache.avro.Schema schema,
TaskContextSupplier taskContextSupplier) |
| Modifier and Type | Field and Description |
|---|---|
protected HoodieTable<T,I,K,O> |
BaseActionExecutor.table |
| Constructor and Description |
|---|
BaseActionExecutor(HoodieEngineContext context,
HoodieWriteConfig config,
HoodieTable<T,I,K,O> table,
String instantTime) |
| Constructor and Description |
|---|
CleanActionExecutor(HoodieEngineContext context,
HoodieWriteConfig config,
HoodieTable<T,I,K,O> table,
String instantTime) |
CleanActionExecutor(HoodieEngineContext context,
HoodieWriteConfig config,
HoodieTable<T,I,K,O> table,
String instantTime,
boolean skipLocking) |
CleanPlanActionExecutor(HoodieEngineContext context,
HoodieWriteConfig config,
HoodieTable<T,I,K,O> table,
String instantTime,
Option<Map<String,String>> extraMetadata) |
CleanPlanner(HoodieEngineContext context,
HoodieTable<T,I,K,O> hoodieTable,
HoodieWriteConfig config) |
| Constructor and Description |
|---|
ClusteringPlanActionExecutor(HoodieEngineContext context,
HoodieWriteConfig config,
HoodieTable<T,I,K,O> table,
String instantTime,
Option<Map<String,String>> extraMetadata) |
| Modifier and Type | Method and Description |
|---|---|
protected HoodieTable<T,I,K,O> |
ClusteringPlanStrategy.getHoodieTable() |
protected HoodieTable<T,I,K,O> |
ClusteringExecutionStrategy.getHoodieTable() |
| Constructor and Description |
|---|
ClusteringExecutionStrategy(HoodieTable table,
HoodieEngineContext engineContext,
HoodieWriteConfig writeConfig) |
ClusteringPlanStrategy(HoodieTable table,
HoodieEngineContext engineContext,
HoodieWriteConfig writeConfig) |
PartitionAwareClusteringPlanStrategy(HoodieTable table,
HoodieEngineContext engineContext,
HoodieWriteConfig writeConfig) |
| Modifier and Type | Method and Description |
|---|---|
abstract HoodieWriteMetadata<O> |
BaseBulkInsertHelper.bulkInsert(I inputRecords,
String instantTime,
HoodieTable<T,I,K,O> table,
HoodieWriteConfig config,
BaseCommitActionExecutor<T,I,K,O,R> executor,
boolean performDedupe,
Option<BulkInsertPartitioner> userDefinedBulkInsertPartitioner)
Mark instant as inflight, write input records, update index and return result.
|
abstract O |
BaseBulkInsertHelper.bulkInsert(I inputRecords,
String instantTime,
HoodieTable<T,I,K,O> table,
HoodieWriteConfig config,
boolean performDedupe,
Option<BulkInsertPartitioner> userDefinedBulkInsertPartitioner,
boolean addMetadataFields,
int parallelism,
WriteHandleFactory writeHandleFactory)
Only write input records.
|
I |
BaseWriteHelper.combineOnCondition(boolean condition,
I records,
int parallelism,
HoodieTable<T,I,K,O> table) |
HoodieData<HoodieKey> |
HoodieDeleteHelper.deduplicateKeys(HoodieData<HoodieKey> keys,
HoodieTable<T,HoodieData<HoodieRecord<T>>,HoodieData<HoodieKey>,HoodieData<WriteStatus>> table,
int parallelism) |
abstract K |
BaseDeleteHelper.deduplicateKeys(K keys,
HoodieTable<T,I,K,O> table,
int parallelism)
Deduplicate Hoodie records, using the given deduplication function.
|
I |
BaseWriteHelper.deduplicateRecords(I records,
HoodieTable<T,I,K,O> table,
int parallelism)
Deduplicate Hoodie records, using the given deduplication function.
|
HoodieWriteMetadata<HoodieData<WriteStatus>> |
HoodieDeleteHelper.execute(String instantTime,
HoodieData<HoodieKey> keys,
HoodieEngineContext context,
HoodieWriteConfig config,
HoodieTable<T,HoodieData<HoodieRecord<T>>,HoodieData<HoodieKey>,HoodieData<WriteStatus>> table,
BaseCommitActionExecutor<T,HoodieData<HoodieRecord<T>>,HoodieData<HoodieKey>,HoodieData<WriteStatus>,R> deleteExecutor) |
abstract HoodieWriteMetadata<O> |
BaseDeleteHelper.execute(String instantTime,
K keys,
HoodieEngineContext context,
HoodieWriteConfig config,
HoodieTable<T,I,K,O> table,
BaseCommitActionExecutor<T,I,K,O,R> deleteExecutor) |
protected Iterator<org.apache.avro.generic.GenericRecord> |
BaseMergeHelper.getMergingIterator(HoodieTable<T,I,K,O> table,
HoodieMergeHandle<T,I,K,O> mergeHandle,
HoodieBaseFile baseFile,
HoodieFileReader<org.apache.avro.generic.GenericRecord> reader,
org.apache.avro.Schema readSchema,
boolean externalSchemaTransformation)
Create Parquet record iterator that provides a stitched view of record read from skeleton and bootstrap file.
|
void |
HoodieMergeHelper.runMerge(HoodieTable<T,HoodieData<HoodieRecord<T>>,HoodieData<HoodieKey>,HoodieData<WriteStatus>> table,
HoodieMergeHandle<T,HoodieData<HoodieRecord<T>>,HoodieData<HoodieKey>,HoodieData<WriteStatus>> mergeHandle) |
abstract void |
BaseMergeHelper.runMerge(HoodieTable<T,I,K,O> table,
HoodieMergeHandle<T,I,K,O> upsertHandle)
Read records from previous version of base file and merge.
|
protected HoodieData<HoodieRecord<T>> |
HoodieWriteHelper.tag(HoodieData<HoodieRecord<T>> dedupedRecords,
HoodieEngineContext context,
HoodieTable<T,HoodieData<HoodieRecord<T>>,HoodieData<HoodieKey>,HoodieData<WriteStatus>> table) |
protected abstract I |
BaseWriteHelper.tag(I dedupedRecords,
HoodieEngineContext context,
HoodieTable<T,I,K,O> table) |
HoodieWriteMetadata<O> |
BaseWriteHelper.write(String instantTime,
I inputRecords,
HoodieEngineContext context,
HoodieTable<T,I,K,O> table,
boolean shouldCombine,
int shuffleParallelism,
BaseCommitActionExecutor<T,I,K,O,R> executor,
WriteOperationType operationType) |
| Constructor and Description |
|---|
BaseCommitActionExecutor(HoodieEngineContext context,
HoodieWriteConfig config,
HoodieTable<T,I,K,O> table,
String instantTime,
WriteOperationType operationType,
Option<Map<String,String>> extraMetadata) |
| Modifier and Type | Method and Description |
|---|---|
HoodieData<WriteStatus> |
HoodieCompactor.compact(HoodieEngineContext context,
HoodieCompactionPlan compactionPlan,
HoodieTable table,
HoodieWriteConfig config,
String compactionInstantTime,
HoodieCompactionHandler compactionHandler)
Execute compaction operations and report back status.
|
void |
CompactHelpers.completeInflightCompaction(HoodieTable table,
String compactionCommitTime,
HoodieCommitMetadata commitMetadata) |
HoodieCommitMetadata |
CompactHelpers.createCompactionMetadata(HoodieTable table,
String compactionInstantTime,
HoodieData<WriteStatus> writeStatuses,
String schema) |
abstract void |
HoodieCompactor.preCompact(HoodieTable table,
HoodieTimeline pendingCompactionTimeline,
String compactionInstantTime)
Handles the compaction timeline based on the compaction instant before actual compaction.
|
| Constructor and Description |
|---|
RunCompactionActionExecutor(HoodieEngineContext context,
HoodieWriteConfig config,
HoodieTable table,
String instantTime,
HoodieCompactor compactor,
HoodieCompactionHandler compactionHandler) |
ScheduleCompactionActionExecutor(HoodieEngineContext context,
HoodieWriteConfig config,
HoodieTable<T,I,K,O> table,
String instantTime,
Option<Map<String,String>> extraMetadata,
HoodieCompactor compactor) |
| Constructor and Description |
|---|
RunIndexActionExecutor(HoodieEngineContext context,
HoodieWriteConfig config,
HoodieTable<T,I,K,O> table,
String instantTime) |
ScheduleIndexActionExecutor(HoodieEngineContext context,
HoodieWriteConfig config,
HoodieTable<T,I,K,O> table,
String instantTime,
List<MetadataPartitionType> partitionIndexTypes) |
| Constructor and Description |
|---|
BaseRestoreActionExecutor(HoodieEngineContext context,
HoodieWriteConfig config,
HoodieTable<T,I,K,O> table,
String instantTime,
String restoreInstantTime) |
CopyOnWriteRestoreActionExecutor(HoodieEngineContext context,
HoodieWriteConfig config,
HoodieTable table,
String instantTime,
String restoreInstantTime) |
MergeOnReadRestoreActionExecutor(HoodieEngineContext context,
HoodieWriteConfig config,
HoodieTable<T,I,K,O> table,
String instantTime,
String restoreInstantTime) |
| Modifier and Type | Field and Description |
|---|---|
protected HoodieTable<?,?,?,?> |
MarkerBasedRollbackStrategy.table |
protected HoodieTable |
ListingBasedRollbackStrategy.table |
| Modifier and Type | Method and Description |
|---|---|
static List<ListingBasedRollbackRequest> |
RollbackUtils.generateRollbackRequestsUsingFileListingMOR(HoodieInstant instantToRollback,
HoodieTable table,
HoodieEngineContext context)
Generate all rollback requests that we need to perform for rolling back this action without actually performing rolling back for MOR table type.
|
| Constructor and Description |
|---|
BaseRollbackActionExecutor(HoodieEngineContext context,
HoodieWriteConfig config,
HoodieTable<T,I,K,O> table,
String instantTime,
HoodieInstant instantToRollback,
boolean deleteInstants,
boolean skipLocking) |
BaseRollbackActionExecutor(HoodieEngineContext context,
HoodieWriteConfig config,
HoodieTable<T,I,K,O> table,
String instantTime,
HoodieInstant instantToRollback,
boolean deleteInstants,
boolean skipTimelinePublish,
boolean useMarkerBasedStrategy,
boolean skipLocking) |
BaseRollbackPlanActionExecutor(HoodieEngineContext context,
HoodieWriteConfig config,
HoodieTable<T,I,K,O> table,
String instantTime,
HoodieInstant instantToRollback,
boolean skipTimelinePublish,
boolean shouldRollbackUsingMarkers) |
CopyOnWriteRollbackActionExecutor(HoodieEngineContext context,
HoodieWriteConfig config,
HoodieTable<T,I,K,O> table,
String instantTime,
HoodieInstant commitInstant,
boolean deleteInstants,
boolean skipLocking) |
CopyOnWriteRollbackActionExecutor(HoodieEngineContext context,
HoodieWriteConfig config,
HoodieTable<T,I,K,O> table,
String instantTime,
HoodieInstant commitInstant,
boolean deleteInstants,
boolean skipTimelinePublish,
boolean useMarkerBasedStrategy,
boolean skipLocking) |
ListingBasedRollbackStrategy(HoodieTable table,
HoodieEngineContext context,
HoodieWriteConfig config,
String instantTime) |
MarkerBasedRollbackStrategy(HoodieTable<?,?,?,?> table,
HoodieEngineContext context,
HoodieWriteConfig config,
String instantTime) |
MergeOnReadRollbackActionExecutor(HoodieEngineContext context,
HoodieWriteConfig config,
HoodieTable<T,I,K,O> table,
String instantTime,
HoodieInstant commitInstant,
boolean deleteInstants,
boolean skipLocking) |
MergeOnReadRollbackActionExecutor(HoodieEngineContext context,
HoodieWriteConfig config,
HoodieTable<T,I,K,O> table,
String instantTime,
HoodieInstant commitInstant,
boolean deleteInstants,
boolean skipTimelinePublish,
boolean useMarkerBasedStrategy,
boolean skipLocking) |
RestorePlanActionExecutor(HoodieEngineContext context,
HoodieWriteConfig config,
HoodieTable<T,I,K,O> table,
String instantTime,
String restoreInstantTime) |
| Modifier and Type | Method and Description |
|---|---|
static void |
SavepointHelpers.deleteSavepoint(HoodieTable table,
String savepointTime) |
static void |
SavepointHelpers.validateSavepointPresence(HoodieTable table,
String savepointTime) |
static void |
SavepointHelpers.validateSavepointRestore(HoodieTable table,
String savepointTime) |
| Constructor and Description |
|---|
SavepointActionExecutor(HoodieEngineContext context,
HoodieWriteConfig config,
HoodieTable<T,I,K,O> table,
String instantTime,
String user,
String comment) |
| Modifier and Type | Method and Description |
|---|---|
static WriteMarkers |
WriteMarkersFactory.get(MarkerType markerType,
HoodieTable table,
String instantTime) |
static List<String> |
MarkerBasedRollbackUtils.getAllMarkerPaths(HoodieTable table,
HoodieEngineContext context,
String instant,
int parallelism)
Gets all marker paths.
|
| Constructor and Description |
|---|
DirectWriteMarkers(HoodieTable table,
String instantTime) |
TimelineServerBasedWriteMarkers(HoodieTable table,
String instantTime) |
| Modifier and Type | Method and Description |
|---|---|
HoodieTable |
SupportsUpgradeDowngrade.getTable(HoodieWriteConfig config,
HoodieEngineContext context) |
| Modifier and Type | Method and Description |
|---|---|
protected void |
ZeroToOneUpgradeHandler.recreateMarkers(String commitInstantTime,
HoodieTable table,
HoodieEngineContext context,
int parallelism)
Recreate markers in new format.
|
Copyright © 2022 The Apache Software Foundation. All rights reserved.