| Modifier and Type | Method and Description |
|---|---|
protected HoodieTimeline |
BaseHoodieTableFileIndex.getActiveTimeline() |
| Modifier and Type | Method and Description |
|---|---|
HoodieTimeline |
HoodieFileGroup.getTimeline() |
| Constructor and Description |
|---|
HoodieFileGroup(HoodieFileGroupId fileGroupId,
HoodieTimeline timeline) |
HoodieFileGroup(String partitionPath,
String id,
HoodieTimeline timeline) |
| Modifier and Type | Method and Description |
|---|---|
HoodieTimeline |
HoodieTableMetaClient.getCommitsAndCompactionTimeline()
Get the commit + pending-compaction timeline visible for this table.
|
HoodieTimeline |
HoodieTableMetaClient.getCommitsTimeline()
Get the commit timeline visible for this table.
|
HoodieTimeline |
HoodieTableMetaClient.getCommitTimeline()
Get the compacted commit timeline visible for this table.
|
| Modifier and Type | Method and Description |
|---|---|
static HoodieTimeline |
IncrementalQueryAnalyzer.filterInstantsAsPerUserConfigs(HoodieTableMetaClient metaClient,
HoodieTimeline timeline,
boolean skipCompaction,
boolean skipClustering,
boolean skipInsertOverwrite)
Filters out the unnecessary instants as per user specified configs.
|
HoodieTimeline |
IncrementalQueryAnalyzer.QueryContext.getActiveTimeline() |
HoodieTimeline |
IncrementalQueryAnalyzer.QueryContext.getArchivedTimeline() |
| Modifier and Type | Method and Description |
|---|---|
static IncrementalQueryAnalyzer.QueryContext |
IncrementalQueryAnalyzer.QueryContext.create(String startInstant,
String endInstant,
List<String> instants,
List<HoodieInstant> archivedInstants,
List<HoodieInstant> activeInstants,
HoodieTimeline activeTimeline,
HoodieTimeline archivedTimeline) |
static HoodieTimeline |
IncrementalQueryAnalyzer.filterInstantsAsPerUserConfigs(HoodieTableMetaClient metaClient,
HoodieTimeline timeline,
boolean skipCompaction,
boolean skipClustering,
boolean skipInsertOverwrite)
Filters out the unnecessary instants as per user specified configs.
|
| Modifier and Type | Class and Description |
|---|---|
class |
HoodieActiveTimeline
Represents the Active Timeline for the Hoodie table.
|
class |
HoodieArchivedTimeline
Represents the Archived Timeline for the Hoodie table.
|
class |
HoodieDefaultTimeline
HoodieDefaultTimeline is a default implementation of the HoodieTimeline.
|
| Modifier and Type | Method and Description |
|---|---|
static HoodieTimeline |
TimelineUtils.concatTimeline(HoodieTimeline timeline1,
HoodieTimeline timeline2,
HoodieTableMetaClient metaClient)
Concat two timelines timeline1 and timeline2 to build a new timeline.
|
HoodieTimeline |
HoodieDefaultTimeline.filter(Predicate<HoodieInstant> filter) |
HoodieTimeline |
HoodieTimeline.filter(Predicate<HoodieInstant> filter)
Custom Filter of Instants.
|
HoodieTimeline |
HoodieDefaultTimeline.filterCompletedAndCompactionInstants() |
HoodieTimeline |
HoodieTimeline.filterCompletedAndCompactionInstants()
Filter this timeline to just include the completed + compaction (inflight + requested) instants A RT filesystem
view is constructed with this timeline so that file-slice after pending compaction-requested instant-time is also
considered valid.
|
HoodieTimeline |
HoodieDefaultTimeline.filterCompletedIndexTimeline() |
HoodieTimeline |
HoodieTimeline.filterCompletedIndexTimeline()
Filter this timeline to just include completed index instants.
|
HoodieTimeline |
HoodieDefaultTimeline.filterCompletedInstants() |
HoodieTimeline |
HoodieTimeline.filterCompletedInstants()
Filter this timeline to just include the completed instants.
|
HoodieTimeline |
HoodieTimeline.filterCompletedInstantsOrRewriteTimeline()
Timeline to just include completed commits or all rewrites like compaction, logcompaction and replace actions.
|
HoodieTimeline |
HoodieDefaultTimeline.filterCompletedOrMajorOrMinorCompactionInstants() |
HoodieTimeline |
HoodieTimeline.filterCompletedOrMajorOrMinorCompactionInstants() |
HoodieTimeline |
HoodieDefaultTimeline.filterInflights() |
HoodieTimeline |
HoodieTimeline.filterInflights()
Filter this timeline to just include the in-flights.
|
HoodieTimeline |
HoodieDefaultTimeline.filterInflightsAndRequested() |
HoodieTimeline |
HoodieTimeline.filterInflightsAndRequested()
Filter this timeline to include requested and in-flights.
|
HoodieTimeline |
HoodieDefaultTimeline.filterPendingCompactionTimeline() |
HoodieTimeline |
HoodieTimeline.filterPendingCompactionTimeline()
Filter this timeline to just include requested and inflight compaction instants.
|
HoodieTimeline |
HoodieDefaultTimeline.filterPendingExcludingCompaction() |
HoodieTimeline |
HoodieTimeline.filterPendingExcludingCompaction()
Filter this timeline to just include the in-flights excluding compaction instants.
|
HoodieTimeline |
HoodieDefaultTimeline.filterPendingExcludingLogCompaction() |
HoodieTimeline |
HoodieTimeline.filterPendingExcludingLogCompaction()
Filter this timeline to just include the in-flights excluding logcompaction instants.
|
HoodieTimeline |
HoodieDefaultTimeline.filterPendingExcludingMajorAndMinorCompaction() |
HoodieTimeline |
HoodieTimeline.filterPendingExcludingMajorAndMinorCompaction()
Filter this timeline to just include the in-flights excluding major and minor compaction instants.
|
HoodieTimeline |
HoodieDefaultTimeline.filterPendingIndexTimeline() |
HoodieTimeline |
HoodieTimeline.filterPendingIndexTimeline()
Filter this timeline to just include requested and inflight index instants.
|
HoodieTimeline |
HoodieDefaultTimeline.filterPendingLogCompactionTimeline() |
HoodieTimeline |
HoodieTimeline.filterPendingLogCompactionTimeline()
Filter this timeline to just include requested and inflight log compaction instants.
|
HoodieTimeline |
HoodieDefaultTimeline.filterPendingMajorOrMinorCompactionTimeline()
Compaction and logcompaction operation on MOR table is called major and minor compaction respectively.
|
HoodieTimeline |
HoodieTimeline.filterPendingMajorOrMinorCompactionTimeline()
Filter this timeline to just include requested and inflight from both major and minor compaction instants.
|
HoodieTimeline |
HoodieDefaultTimeline.filterPendingReplaceTimeline() |
HoodieTimeline |
HoodieTimeline.filterPendingReplaceTimeline()
Filter this timeline to just include requested and inflight replacecommit instants.
|
HoodieTimeline |
HoodieDefaultTimeline.filterPendingRollbackTimeline() |
HoodieTimeline |
HoodieTimeline.filterPendingRollbackTimeline()
Filter this timeline to include pending rollbacks.
|
HoodieTimeline |
HoodieDefaultTimeline.filterRequestedRollbackTimeline() |
HoodieTimeline |
HoodieTimeline.filterRequestedRollbackTimeline()
Filter this timeline for requested rollbacks.
|
HoodieTimeline |
HoodieDefaultTimeline.findInstantsAfter(String instantTime) |
HoodieTimeline |
HoodieTimeline.findInstantsAfter(String instantTime)
Create a new Timeline with all the instants after startTs.
|
HoodieTimeline |
HoodieTimeline.findInstantsAfter(String instantTime,
int numCommits)
Create a new Timeline with all the instants after startTs.
|
HoodieTimeline |
HoodieTimeline.findInstantsAfterOrEquals(String commitTime,
int numCommits)
Create a new Timeline with all the instants after startTs.
|
HoodieTimeline |
HoodieTimeline.findInstantsBefore(String instantTime)
Create a new Timeline with all instants before specified time.
|
HoodieTimeline |
HoodieTimeline.findInstantsBeforeOrEquals(String instantTime)
Create new timeline with all instants before or equals specified time.
|
HoodieTimeline |
HoodieTimeline.findInstantsInClosedRange(String startTs,
String endTs)
Create a new Timeline with instants after or equals startTs and before or on endTs.
|
HoodieTimeline |
HoodieTimeline.findInstantsInRange(String startTs,
String endTs)
Create a new Timeline with instants after startTs and before or on endTs.
|
HoodieTimeline |
HoodieTimeline.findInstantsInRangeByCompletionTime(String startTs,
String endTs)
`
Create a new Timeline with instants after startTs and before or on endTs
by state transition timestamp of actions.
|
HoodieTimeline |
HoodieTimeline.findInstantsModifiedAfterByCompletionTime(String instantTime)
Create new timeline with all instants that were modified after specified time.
|
HoodieTimeline |
HoodieDefaultTimeline.getAllCommitsTimeline()
Get all instants (commits, delta commits, compaction, clean, savepoint, rollback, replace commits, index) that result in actions,
in the active timeline.
|
HoodieTimeline |
HoodieDefaultTimeline.getCleanerTimeline()
Get only the cleaner action (inflight and completed) in the active timeline.
|
HoodieTimeline |
HoodieDefaultTimeline.getCommitAndReplaceTimeline()
Get only pure commit and replace commits (inflight and completed) in the active timeline.
|
HoodieTimeline |
HoodieDefaultTimeline.getCommitsAndCompactionTimeline()
Get all instants (commits, delta commits, replace, compaction) that produce new data or merge file, in the active timeline.
|
HoodieTimeline |
HoodieDefaultTimeline.getCommitsTimeline()
Get all instants (commits, delta commits) that produce new data, in the active timeline.
|
static HoodieTimeline |
TimelineUtils.getCommitsTimelineAfter(HoodieTableMetaClient metaClient,
String exclusiveStartInstantTime,
Option<String> lastMaxCompletionTime)
Returns a Hudi timeline with commits after the given instant time (exclusive).
|
HoodieTimeline |
HoodieDefaultTimeline.getCommitTimeline()
Get only pure commits (inflight and completed) in the active timeline.
|
HoodieTimeline |
HoodieDefaultTimeline.getCompletedReplaceTimeline() |
HoodieTimeline |
HoodieTimeline.getCompletedReplaceTimeline()
Timeline to just include replace instants that have valid (commit/deltacommit) actions.
|
HoodieTimeline |
HoodieDefaultTimeline.getContiguousCompletedWriteTimeline() |
HoodieTimeline |
HoodieTimeline.getContiguousCompletedWriteTimeline()
Timeline to just include commits (commit/deltacommit), compaction and replace actions that are completed and contiguous.
|
HoodieTimeline |
HoodieDefaultTimeline.getDeltaCommitTimeline()
Get only the delta commits (inflight and completed) in the active timeline.
|
HoodieTimeline |
HoodieDefaultTimeline.getRestoreTimeline()
Get only the restore action (inflight and completed) in the active timeline.
|
HoodieTimeline |
HoodieDefaultTimeline.getRollbackAndRestoreTimeline()
Get only the rollback and restore action (inflight and completed) in the active timeline.
|
HoodieTimeline |
HoodieDefaultTimeline.getRollbackTimeline()
Get only the rollback action (inflight and completed) in the active timeline.
|
HoodieTimeline |
HoodieDefaultTimeline.getSavePointTimeline()
Get only the save point action (inflight and completed) in the active timeline.
|
HoodieTimeline |
HoodieDefaultTimeline.getTimelineOfActions(Set<String> actions)
Get a timeline of a specific set of actions.
|
HoodieTimeline |
HoodieTimeline.getWriteTimeline()
Timeline to just include commits (commit/deltacommit), compaction and replace actions.
|
static HoodieTimeline |
TimelineUtils.handleHollowCommitIfNeeded(HoodieTimeline completedCommitTimeline,
HoodieTableMetaClient metaClient,
TimelineUtils.HollowCommitHandling handlingMode)
Handles hollow commit as per
HoodieCommonConfig.INCREMENTAL_READ_HANDLE_HOLLOW_COMMIT
and return filtered or non-filtered timeline for incremental query to run against. |
| Modifier and Type | Method and Description |
|---|---|
static HoodieTimeline |
TimelineUtils.concatTimeline(HoodieTimeline timeline1,
HoodieTimeline timeline2,
HoodieTableMetaClient metaClient)
Concat two timelines timeline1 and timeline2 to build a new timeline.
|
static List<String> |
TimelineUtils.getAffectedPartitions(HoodieTimeline timeline)
Returns partitions that have been modified including internal operations such as clean in the passed timeline.
|
static HoodieCommitMetadata |
TimelineUtils.getCommitMetadata(HoodieInstant instant,
HoodieTimeline timeline)
Returns the commit metadata of the given instant.
|
static TimelineDiffHelper.TimelineDiffResult |
TimelineDiffHelper.getNewInstantsForIncrementalSync(HoodieTimeline oldTimeline,
HoodieTimeline newTimeline) |
List<String> |
CompletionTimeQueryView.getStartTimes(HoodieTimeline timeline,
Option<String> rangeStart,
Option<String> rangeEnd,
InstantRange.RangeType rangeType)
Queries the instant start time with given completion time range.
|
List<String> |
CompletionTimeQueryView.getStartTimes(HoodieTimeline timeline,
Option<String> rangeStart,
Option<String> rangeEnd,
InstantRange.RangeType rangeType,
Function<String,String> earliestInstantTimeFunc)
Queries the instant start time with given completion time range.
|
static List<String> |
TimelineUtils.getWrittenPartitions(HoodieTimeline timeline)
Returns partitions that have new data strictly after commitTime.
|
static HoodieTimeline |
TimelineUtils.handleHollowCommitIfNeeded(HoodieTimeline completedCommitTimeline,
HoodieTableMetaClient metaClient,
TimelineUtils.HollowCommitHandling handlingMode)
Handles hollow commit as per
HoodieCommonConfig.INCREMENTAL_READ_HANDLE_HOLLOW_COMMIT
and return filtered or non-filtered timeline for incremental query to run against. |
| Modifier and Type | Method and Description |
|---|---|
static HoodieTimeline |
TimelineDTO.toTimeline(TimelineDTO dto,
HoodieTableMetaClient metaClient) |
| Modifier and Type | Method and Description |
|---|---|
static TimelineDTO |
TimelineDTO.fromTimeline(HoodieTimeline timeline) |
static HoodieFileGroup |
FileGroupDTO.toFileGroup(FileGroupDTO dto,
HoodieTimeline fgTimeline) |
| Modifier and Type | Method and Description |
|---|---|
HoodieTimeline |
TableFileSystemView.getTimeline()
Timeline corresponding to the view.
|
HoodieTimeline |
IncrementalTimelineSyncFileSystemView.getTimeline() |
HoodieTimeline |
AbstractTableFileSystemView.getTimeline() |
HoodieTimeline |
PriorityBasedFileSystemView.getTimeline() |
HoodieTimeline |
RemoteHoodieTableFileSystemView.getTimeline() |
HoodieTimeline |
AbstractTableFileSystemView.getVisibleCommitsAndCompactionTimeline()
Return Only Commits and Compaction timeline for building file-groups.
|
| Modifier and Type | Method and Description |
|---|---|
protected List<HoodieFileGroup> |
AbstractTableFileSystemView.buildFileGroups(String partition,
List<StoragePathInfo> statuses,
HoodieTimeline timeline,
boolean addPendingCompactionFileSlice)
Build FileGroups from passed in file-status.
|
protected List<HoodieFileGroup> |
AbstractTableFileSystemView.buildFileGroups(String partition,
Stream<HoodieBaseFile> baseFileStream,
Stream<HoodieLogFile> logFileStream,
HoodieTimeline timeline,
boolean addPendingCompactionFileSlice) |
static HoodieTableFileSystemView |
FileSystemViewManager.createInMemoryFileSystemViewWithTimeline(HoodieEngineContext engineContext,
HoodieTableMetaClient metaClient,
HoodieMetadataConfig metadataConfig,
HoodieTimeline timeline) |
void |
HoodieTableFileSystemView.init(HoodieTableMetaClient metaClient,
HoodieTimeline visibleActiveTimeline) |
protected void |
RocksDbBasedFileSystemView.init(HoodieTableMetaClient metaClient,
HoodieTimeline visibleActiveTimeline) |
protected void |
AbstractTableFileSystemView.init(HoodieTableMetaClient metaClient,
HoodieTimeline visibleActiveTimeline)
Initialize the view.
|
void |
HoodieTableFileSystemView.init(HoodieTableMetaClient metaClient,
HoodieTimeline visibleActiveTimeline,
List<StoragePathInfo> pathInfoList)
Visible for testing
|
protected void |
IncrementalTimelineSyncFileSystemView.refreshTimeline(HoodieTimeline visibleActiveTimeline) |
protected void |
AbstractTableFileSystemView.refreshTimeline(HoodieTimeline visibleActiveTimeline)
Refresh commits timeline.
|
| Constructor and Description |
|---|
HoodieTableFileSystemView(HoodieTableMetaClient metaClient,
HoodieTimeline visibleActiveTimeline)
Create a file system view, as of the given timeline.
|
HoodieTableFileSystemView(HoodieTableMetaClient metaClient,
HoodieTimeline visibleActiveTimeline,
boolean enableIncrementalTimelineSync)
Create a file system view, as of the given timeline.
|
HoodieTableFileSystemView(HoodieTableMetaClient metaClient,
HoodieTimeline visibleActiveTimeline,
List<StoragePathInfo> pathInfoList)
Create a file system view, as of the given timeline, with the provided file statuses.
|
RocksDbBasedFileSystemView(HoodieTableMetaClient metaClient,
HoodieTimeline visibleActiveTimeline,
FileSystemViewStorageConfig config) |
RocksDbBasedFileSystemView(HoodieTableMetaClient metaClient,
HoodieTimeline visibleActiveTimeline,
List<StoragePathInfo> pathInfoList,
FileSystemViewStorageConfig config) |
SpillableMapBasedFileSystemView(HoodieTableMetaClient metaClient,
HoodieTimeline visibleActiveTimeline,
FileSystemViewStorageConfig config,
HoodieCommonConfig commonConfig) |
SpillableMapBasedFileSystemView(HoodieTableMetaClient metaClient,
HoodieTimeline visibleActiveTimeline,
List<StoragePathInfo> pathInfoList,
FileSystemViewStorageConfig config,
HoodieCommonConfig commonConfig) |
| Modifier and Type | Method and Description |
|---|---|
static Option<Pair<HoodieTimeline,HoodieInstant>> |
CompactionUtils.getCompletedDeltaCommitsSinceLatestCompaction(HoodieActiveTimeline activeTimeline)
Returns a pair of (timeline containing the completed delta commits after the latest completed
compaction commit, the completed compaction commit instant), if the latest completed
compaction commit is present; a pair of (timeline containing all the completed delta commits,
the first delta commit instant), if there is no completed compaction commit.
|
static Option<Pair<HoodieTimeline,HoodieInstant>> |
CompactionUtils.getDeltaCommitsSinceLatestCompaction(HoodieActiveTimeline activeTimeline)
Returns a pair of (timeline containing the delta commits after the latest completed
compaction commit, the completed compaction commit instant), if the latest completed
compaction commit is present; a pair of (timeline containing all the delta commits,
the first delta commit instant), if there is no completed compaction commit.
|
static Option<Pair<HoodieTimeline,HoodieInstant>> |
CompactionUtils.getDeltaCommitsSinceLatestCompactionRequest(HoodieActiveTimeline activeTimeline) |
| Modifier and Type | Method and Description |
|---|---|
static Option<Pair<HoodieInstant,HoodieClusteringPlan>> |
ClusteringUtils.getClusteringPlan(HoodieTimeline timeline,
HoodieInstant pendingReplaceInstant)
Get Clustering plan from timeline.
|
static Option<HoodieInstant> |
CleanerUtils.getEarliestCommitToRetain(HoodieTimeline commitsTimeline,
HoodieCleaningPolicy cleaningPolicy,
int commitsRetained,
Instant latestInstant,
int hoursRetained,
HoodieTimelineTimeZone timeZone) |
static Option<String> |
CommitUtils.getValidCheckpointForCurrentWriter(HoodieTimeline timeline,
String checkpointKey,
String keyToLookup)
Process previous commits metadata in the timeline to determine the checkpoint given a checkpoint key.
|
static boolean |
ClusteringUtils.isClusteringInstant(HoodieTimeline timeline,
HoodieInstant replaceInstant)
Checks if the requested, inflight, or completed instant of replacecommit action
is a clustering operation, by checking whether the requested instant contains
a clustering plan.
|
static boolean |
ClusteringUtils.isCompletedClusteringInstant(HoodieInstant instant,
HoodieTimeline timeline) |
static boolean |
ClusteringUtils.isInsertOverwriteInstant(HoodieInstant instant,
HoodieTimeline timeline)
Returns whether the given instant
instant is with insert overwrite operation. |
| Modifier and Type | Method and Description |
|---|---|
static boolean |
HoodieTableMetadataUtil.isIndexingCommit(HoodieTimeline dataIndexTimeline,
String instant)
Checks if a delta commit in metadata table is written by async indexer.
|
| Constructor and Description |
|---|
HoodieMetadataFileSystemView(HoodieEngineContext engineContext,
HoodieTableMetaClient metaClient,
HoodieTimeline visibleActiveTimeline,
HoodieMetadataConfig metadataConfig) |
HoodieMetadataFileSystemView(HoodieTableMetaClient metaClient,
HoodieTimeline visibleActiveTimeline,
HoodieTableMetadata tableMetadata) |
Copyright © 2024 The Apache Software Foundation. All rights reserved.