org.apache.hudi.common.table.timeline

Class HoodieActiveTimeline

java.lang.Object

org.apache.hudi.common.table.timeline.HoodieDefaultTimeline

org.apache.hudi.common.table.timeline.HoodieActiveTimeline

All Implemented Interfaces:

Serializable, HoodieTimeline

public class HoodieActiveTimeline
extends HoodieDefaultTimeline

Represents the Active Timeline for the Hoodie table. Instants for the last 12 hours (configurable) is in the ActiveTimeline and the rest are Archived. ActiveTimeline is a special timeline that allows for creation of instants on the timeline.

The timeline is not automatically reloaded on any mutation operation, clients have to manually call reload() so that they can chain multiple mutations to the timeline and then call reload() once.

This class can be serialized and de-serialized and on de-serialization the FileSystem is re-initialized.

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
protected HoodieTableMetaClient	metaClient
static Set<String>	NOT_PARSABLE_TIMESTAMPS
static Set<String>	VALID_EXTENSIONS_IN_ACTIVE_TIMELINE

Fields inherited from class org.apache.hudi.common.table.timeline.HoodieDefaultTimeline

details

Fields inherited from interface org.apache.hudi.common.table.timeline.HoodieTimeline

CLEAN_ACTION, CLEAN_EXTENSION, COMMIT_ACTION, COMMIT_EXTENSION, COMPACTION_ACTION, COMPLETED_COMPACTION_SUFFIX, COMPLETED_EXTENSION, DELTA_COMMIT_ACTION, DELTA_COMMIT_EXTENSION, EQUALS, FULL_BOOTSTRAP_INSTANT_TS, GREATER_THAN, GREATER_THAN_OR_EQUALS, INDEX_COMMIT_EXTENSION, INDEXING_ACTION, INFLIGHT_CLEAN_EXTENSION, INFLIGHT_COMMIT_EXTENSION, INFLIGHT_COMPACTION_EXTENSION, INFLIGHT_DELTA_COMMIT_EXTENSION, INFLIGHT_EXTENSION, INFLIGHT_INDEX_COMMIT_EXTENSION, INFLIGHT_LOG_COMPACTION_EXTENSION, INFLIGHT_REPLACE_COMMIT_EXTENSION, INFLIGHT_RESTORE_EXTENSION, INFLIGHT_ROLLBACK_EXTENSION, INFLIGHT_SAVE_SCHEMA_ACTION_EXTENSION, INFLIGHT_SAVEPOINT_EXTENSION, INIT_INSTANT_TS, INVALID_INSTANT_TS, LESSER_THAN, LESSER_THAN_OR_EQUALS, LOG_COMPACTION_ACTION, METADATA_BOOTSTRAP_INSTANT_TS, REPLACE_COMMIT_ACTION, REPLACE_COMMIT_EXTENSION, REQUESTED_CLEAN_EXTENSION, REQUESTED_COMMIT_EXTENSION, REQUESTED_COMPACTION_EXTENSION, REQUESTED_COMPACTION_SUFFIX, REQUESTED_DELTA_COMMIT_EXTENSION, REQUESTED_EXTENSION, REQUESTED_INDEX_COMMIT_EXTENSION, REQUESTED_LOG_COMPACTION_EXTENSION, REQUESTED_LOG_COMPACTION_SUFFIX, REQUESTED_REPLACE_COMMIT_EXTENSION, REQUESTED_RESTORE_EXTENSION, REQUESTED_ROLLBACK_EXTENSION, REQUESTED_SAVE_SCHEMA_ACTION_EXTENSION, RESTORE_ACTION, RESTORE_EXTENSION, ROLLBACK_ACTION, ROLLBACK_EXTENSION, SAVE_SCHEMA_ACTION_EXTENSION, SAVEPOINT_ACTION, SAVEPOINT_EXTENSION, SCHEMA_COMMIT_ACTION, VALID_ACTIONS_IN_TIMELINE

Constructor Summary

Constructors

Modifier	Constructor and Description
	HoodieActiveTimeline() Deprecated.
	HoodieActiveTimeline(HoodieTableMetaClient metaClient)
	HoodieActiveTimeline(HoodieTableMetaClient metaClient, boolean applyLayoutFilter)
protected	HoodieActiveTimeline(HoodieTableMetaClient metaClient, Set<String> includedExtensions)
protected	HoodieActiveTimeline(HoodieTableMetaClient metaClient, Set<String> includedExtensions, boolean applyLayoutFilters)

Method Summary

Modifier and Type	Method and Description
void	copyInstant(HoodieInstant instant, StoragePath dstDir)
protected void	createCompleteFileInMetaPath(boolean shouldLock, HoodieInstant instant, Option<byte[]> content)
void	createCompleteInstant(HoodieInstant instant) Create a complete instant and save to storage with a completion time.
protected void	createFileInMetaPath(String filename, Option<byte[]> content, boolean allowOverwrite)
void	createNewInstant(HoodieInstant instant) Create a pending instant and save to storage.
static String	createNewInstantTime(boolean shouldLock, TimeGenerator timeGenerator) Returns next instant time in the correct format.
static String	createNewInstantTime(boolean shouldLock, TimeGenerator timeGenerator, long milliseconds) Returns next instant time in the correct format.
void	createRequestedReplaceCommit(String instantTime, String actionType)
void	deleteCompactionRequested(HoodieInstant instant)
void	deleteCompletedRollback(HoodieInstant instant)
void	deleteEmptyInstantIfExists(HoodieInstant instant)
void	deleteInflight(HoodieInstant instant)
protected void	deleteInstantFile(HoodieInstant instant)
static void	deleteInstantFile(HoodieStorage storage, StoragePath metaPath, HoodieInstant instant)
void	deleteInstantFileIfExists(HoodieInstant instant) Note: This method should only be used in the case that delete requested/inflight instant or empty clean instant, and completed commit instant in an archive operation.
void	deletePending(HoodieInstant instant)
static String	formatDate(Date timestamp) Format the Date to a String representing the timestamp of a Hoodie Instant.
Option<byte[]>	getInstantDetails(HoodieInstant instant) Read the completed instant details.
protected String	getInstantFileName(HoodieInstant instant) Many callers might not pass completionTime, here we have to search timeline to get completionTime, the impact should be minor since 1.
Option<Pair<HoodieInstant,HoodieCommitMetadata>>	getLastCommitMetadataWithValidData() Get the last instant with valid data, and convert this to HoodieCommitMetadata
Option<Pair<HoodieInstant,HoodieCommitMetadata>>	getLastCommitMetadataWithValidSchema() Returns most recent instant having valid schema in its HoodieCommitMetadata
static Date	parseDateFromInstantTime(String timestamp) Parse the timestamp of an Instant and return a Date.
static Option<Date>	parseDateFromInstantTimeSafely(String timestamp) The same parsing method as above, but this method will mute ParseException.
Option<byte[]>	readCleanerInfoAsBytes(HoodieInstant instant)
Option<byte[]>	readCompactionPlanAsBytes(HoodieInstant instant)
protected Option<byte[]>	readDataFromPath(StoragePath detailPath)
Option<byte[]>	readIndexPlanAsBytes(HoodieInstant instant)
Option<byte[]>	readRestoreInfoAsBytes(HoodieInstant instant)
Option<byte[]>	readRollbackInfoAsBytes(HoodieInstant instant)
HoodieActiveTimeline	reload()
protected void	revertCompleteToInflight(HoodieInstant completed, HoodieInstant inflight)
HoodieInstant	revertIndexInflightToRequested(HoodieInstant inflightInstant) Revert index instant state from inflight to requested.
HoodieInstant	revertInstantFromInflightToRequested(HoodieInstant inflightInstant) Revert instant state from inflight to requested.
HoodieInstant	revertLogCompactionInflightToRequested(HoodieInstant inflightInstant) TODO: This method is not needed, since log compaction plan is not a immutable plan.
HoodieInstant	revertToInflight(HoodieInstant instant)
void	saveAsComplete(boolean shouldLock, HoodieInstant instant, Option<byte[]> data)
void	saveAsComplete(HoodieInstant instant, Option<byte[]> data)
void	saveToCleanRequested(HoodieInstant instant, Option<byte[]> content)
void	saveToCompactionRequested(HoodieInstant instant, Option<byte[]> content)
void	saveToCompactionRequested(HoodieInstant instant, Option<byte[]> content, boolean overwrite)
void	saveToLogCompactionRequested(HoodieInstant instant, Option<byte[]> content)
void	saveToLogCompactionRequested(HoodieInstant instant, Option<byte[]> content, boolean overwrite)
void	saveToPendingIndexAction(HoodieInstant instant, Option<byte[]> content) Save content for inflight/requested index instant.
void	saveToPendingReplaceCommit(HoodieInstant instant, Option<byte[]> content) Saves content for requested REPLACE instant.
void	saveToRestoreRequested(HoodieInstant instant, Option<byte[]> content)
void	saveToRollbackRequested(HoodieInstant instant, Option<byte[]> content)
HoodieInstant	transitionCleanInflightToComplete(boolean shouldLock, HoodieInstant inflightInstant, Option<byte[]> data) Transition Clean State from inflight to Committed.
HoodieInstant	transitionCleanRequestedToInflight(HoodieInstant requestedInstant, Option<byte[]> data) Transition Clean State from requested to inflight.
HoodieInstant	transitionCompactionInflightToComplete(boolean shouldLock, HoodieInstant inflightInstant, Option<byte[]> data) Transition Compaction State from inflight to Committed.
HoodieInstant	transitionCompactionRequestedToInflight(HoodieInstant requestedInstant) Transition Compaction State from requested to inflight.
HoodieInstant	transitionIndexInflightToComplete(boolean shouldLock, HoodieInstant inflightInstant, Option<byte[]> data) Transition index instant state from inflight to completed.
HoodieInstant	transitionIndexRequestedToInflight(HoodieInstant requestedInstant, Option<byte[]> data) Transition index instant state from requested to inflight.
HoodieInstant	transitionLogCompactionInflightToComplete(boolean shouldLock, HoodieInstant inflightInstant, Option<byte[]> data) Transition Log Compaction State from inflight to Committed.
HoodieInstant	transitionLogCompactionRequestedToInflight(HoodieInstant requestedInstant) Transition LogCompaction State from requested to inflight.
protected void	transitionPendingState(HoodieInstant fromInstant, HoodieInstant toInstant, Option<byte[]> data, boolean allowRedundantTransitions)
HoodieInstant	transitionReplaceInflightToComplete(boolean shouldLock, HoodieInstant inflightInstant, Option<byte[]> data) Transition replace inflight to Committed.
HoodieInstant	transitionReplaceRequestedToInflight(HoodieInstant requestedInstant, Option<byte[]> data) Transition replace requested file to replace inflight.
void	transitionRequestedToInflight(HoodieInstant requested, Option<byte[]> content)
void	transitionRequestedToInflight(HoodieInstant requested, Option<byte[]> content, boolean allowRedundantTransitions)
void	transitionRequestedToInflight(String commitType, String inFlightInstant)
HoodieInstant	transitionRestoreRequestedToInflight(HoodieInstant requestedInstant) Transition Restore State from requested to inflight.
HoodieInstant	transitionRollbackInflightToComplete(boolean shouldLock, HoodieInstant inflightInstant, Option<byte[]> data) Transition Rollback State from inflight to Committed.
HoodieInstant	transitionRollbackRequestedToInflight(HoodieInstant requestedInstant) Transition Rollback State from requested to inflight.
protected void	transitionStateToComplete(boolean shouldLock, HoodieInstant fromInstant, HoodieInstant toInstant, Option<byte[]> data)

Methods inherited from class org.apache.hudi.common.table.timeline.HoodieDefaultTimeline

appendInstants, containsInstant, containsInstant, containsOrBeforeTimelineStarts, countInstants, empty, filter, filterCompletedAndCompactionInstants, filterCompletedIndexTimeline, filterCompletedInstants, filterCompletedInstantsOrRewriteTimeline, filterCompletedOrMajorOrMinorCompactionInstants, filterInflights, filterInflightsAndRequested, filterInstantsByAction, filterPendingCompactionTimeline, filterPendingExcludingCompaction, filterPendingExcludingLogCompaction, filterPendingExcludingMajorAndMinorCompaction, filterPendingIndexTimeline, filterPendingLogCompactionTimeline, filterPendingMajorOrMinorCompactionTimeline, filterPendingReplaceTimeline, filterPendingRollbackTimeline, filterRequestedRollbackTimeline, findInstantBefore, findInstantsAfter, findInstantsAfter, findInstantsAfterOrEquals, findInstantsBefore, findInstantsBeforeOrEquals, findInstantsInClosedRange, findInstantsInRange, findInstantsInRangeByCompletionTime, findInstantsModifiedAfterByCompletionTime, firstInstant, firstInstant, getAllCommitsTimeline, getCleanerTimeline, getCommitAndReplaceTimeline, getCommitsAndCompactionTimeline, getCommitsTimeline, getCommitTimeline, getCompletedReplaceTimeline, getContiguousCompletedWriteTimeline, getDeltaCommitTimeline, getFirstNonSavepointCommit, getFirstPendingClusterInstant, getInstants, getInstantsAsStream, getInstantsOrderedByCompletionTime, getLastClusteringInstant, getLastPendingClusterInstant, getRestoreTimeline, getReverseOrderedInstants, getRollbackAndRestoreTimeline, getRollbackTimeline, getSavePointTimeline, getTimelineHash, getTimelineOfActions, getWriteTimeline, isBeforeTimelineStarts, isEmpty, isPendingClusterInstant, lastInstant, mergeTimeline, nthFromLastInstant, nthInstant, setInstants, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.hudi.common.table.timeline.HoodieTimeline

compareTimestamps, getCleanInflightInstant, getCleanRequestedInstant, getCommitFromCommitFile, getCompactionInflightInstant, getCompactionRequestedInstant, getIndexInflightInstant, getIndexRequestedInstant, getInflightInstant, getLogCompactionInflightInstant, getLogCompactionRequestedInstant, getReplaceCommitInflightInstant, getReplaceCommitRequestedInstant, getRequestedInstant, getRestoreRequestedInstant, getRollbackRequestedInstant, isInClosedOpenRange, isInClosedRange, isInRange, makeCleanerFileName, makeCommitFileName, makeDeltaFileName, makeFileNameAsComplete, makeFileNameAsInflight, makeIndexCommitFileName, makeInflightCleanerFileName, makeInflightCommitFileName, makeInflightCompactionFileName, makeInflightDeltaFileName, makeInflightIndexFileName, makeInflightLogCompactionFileName, makeInflightReplaceFileName, makeInflightRestoreFileName, makeInflightRollbackFileName, makeInflightSavePointFileName, makeInflightSchemaFileName, makeReplaceFileName, makeRequestedCleanerFileName, makeRequestedCommitFileName, makeRequestedCompactionFileName, makeRequestedDeltaFileName, makeRequestedIndexFileName, makeRequestedLogCompactionFileName, makeRequestedReplaceFileName, makeRequestedRestoreFileName, makeRequestedRollbackFileName, makeRequestSchemaFileName, makeRestoreFileName, makeRollbackFileName, makeSavePointFileName, makeSchemaFileName, maxInstant, minInstant

Field Detail

VALID_EXTENSIONS_IN_ACTIVE_TIMELINE

public static final Set<String> VALID_EXTENSIONS_IN_ACTIVE_TIMELINE

NOT_PARSABLE_TIMESTAMPS

public static final Set<String> NOT_PARSABLE_TIMESTAMPS

metaClient

protected HoodieTableMetaClient metaClient

Constructor Detail

HoodieActiveTimeline

protected HoodieActiveTimeline(HoodieTableMetaClient metaClient,
                               Set<String> includedExtensions)

HoodieActiveTimeline

protected HoodieActiveTimeline(HoodieTableMetaClient metaClient,
                               Set<String> includedExtensions,
                               boolean applyLayoutFilters)

HoodieActiveTimeline

public HoodieActiveTimeline(HoodieTableMetaClient metaClient)

HoodieActiveTimeline

public HoodieActiveTimeline(HoodieTableMetaClient metaClient,
                            boolean applyLayoutFilter)

HoodieActiveTimeline

@Deprecated
public HoodieActiveTimeline()

Deprecated.

For serialization and de-serialization only.

Method Detail

parseDateFromInstantTime

public static Date parseDateFromInstantTime(String timestamp)
                                     throws ParseException

Parse the timestamp of an Instant and return a Date. Throw ParseException if timestamp is not valid format as HoodieInstantTimeGenerator.SECS_INSTANT_TIMESTAMP_FORMAT.

Parameters:

timestamp - a timestamp String which follow pattern as HoodieInstantTimeGenerator.SECS_INSTANT_TIMESTAMP_FORMAT.

Returns:

Date of instant timestamp

Throws:

ParseException

parseDateFromInstantTimeSafely

public static Option<Date> parseDateFromInstantTimeSafely(String timestamp)

The same parsing method as above, but this method will mute ParseException. If the given timestamp is invalid, returns Option.empty. Or a corresponding Date value if these timestamp strings are provided HoodieTimeline.INIT_INSTANT_TS, HoodieTimeline.METADATA_BOOTSTRAP_INSTANT_TS, HoodieTimeline.FULL_BOOTSTRAP_INSTANT_TS. This method is useful when parsing timestamp for metrics

Parameters:

timestamp - a timestamp String which follow pattern as HoodieInstantTimeGenerator.SECS_INSTANT_TIMESTAMP_FORMAT.

Returns:

Option<Date> of instant timestamp, Option.empty if invalid timestamp

formatDate

public static String formatDate(Date timestamp)

Format the Date to a String representing the timestamp of a Hoodie Instant.

createNewInstantTime

public static String createNewInstantTime(boolean shouldLock,
                                          TimeGenerator timeGenerator)

Returns next instant time in the correct format. Ensures each instant time is at least 1 millisecond apart since we create instant times at millisecond granularity.

Parameters:

shouldLock - whether the lock should be enabled to get the instant time.

timeGenerator - TimeGenerator used to generate the instant time.

createNewInstantTime

public static String createNewInstantTime(boolean shouldLock,
                                          TimeGenerator timeGenerator,
                                          long milliseconds)

Returns next instant time in the correct format. Ensures each instant time is at least 1 millisecond apart since we create instant times at millisecond granularity.

Parameters:

shouldLock - whether the lock should be enabled to get the instant time.

timeGenerator - TimeGenerator used to generate the instant time.

milliseconds - Milliseconds to add to current time while generating the new instant time

createCompleteInstant

public void createCompleteInstant(HoodieInstant instant)

Create a complete instant and save to storage with a completion time.

Parameters:

instant - the complete instant.

createNewInstant

public void createNewInstant(HoodieInstant instant)

Create a pending instant and save to storage.

Parameters:

instant - the pending instant.

createRequestedReplaceCommit

public void createRequestedReplaceCommit(String instantTime,
                                         String actionType)

saveAsComplete

public void saveAsComplete(HoodieInstant instant,
                           Option<byte[]> data)

saveAsComplete

public void saveAsComplete(boolean shouldLock,
                           HoodieInstant instant,
                           Option<byte[]> data)

revertToInflight

public HoodieInstant revertToInflight(HoodieInstant instant)

deleteInflight

public void deleteInflight(HoodieInstant instant)

deletePending

public void deletePending(HoodieInstant instant)

deleteCompletedRollback

public void deleteCompletedRollback(HoodieInstant instant)

deleteInstantFile

public static void deleteInstantFile(HoodieStorage storage,
                                     StoragePath metaPath,
                                     HoodieInstant instant)

deleteEmptyInstantIfExists

public void deleteEmptyInstantIfExists(HoodieInstant instant)

deleteCompactionRequested

public void deleteCompactionRequested(HoodieInstant instant)

deleteInstantFileIfExists

public void deleteInstantFileIfExists(HoodieInstant instant)

Note: This method should only be used in the case that delete requested/inflight instant or empty clean instant, and completed commit instant in an archive operation.

deleteInstantFile

protected void deleteInstantFile(HoodieInstant instant)

getInstantFileName

protected String getInstantFileName(HoodieInstant instant)

Many callers might not pass completionTime, here we have to search timeline to get completionTime, the impact should be minor since 1. It appeals only tests pass instant without completion time 2. we already holds all instants in memory, the cost should be minor.

TODO: [HUDI-6885] Depreciate HoodieActiveTimeline#getInstantFileName and fix related tests.

getInstantDetails

public Option<byte[]> getInstantDetails(HoodieInstant instant)

Description copied from interface: HoodieTimeline

Read the completed instant details.

Specified by:

getInstantDetails in interface HoodieTimeline

Overrides:

getInstantDetails in class HoodieDefaultTimeline

getLastCommitMetadataWithValidSchema

public Option<Pair<HoodieInstant,HoodieCommitMetadata>> getLastCommitMetadataWithValidSchema()

Returns most recent instant having valid schema in its HoodieCommitMetadata

getLastCommitMetadataWithValidData

public Option<Pair<HoodieInstant,HoodieCommitMetadata>> getLastCommitMetadataWithValidData()

Get the last instant with valid data, and convert this to HoodieCommitMetadata

readCleanerInfoAsBytes

public Option<byte[]> readCleanerInfoAsBytes(HoodieInstant instant)

readRollbackInfoAsBytes

public Option<byte[]> readRollbackInfoAsBytes(HoodieInstant instant)

readRestoreInfoAsBytes

public Option<byte[]> readRestoreInfoAsBytes(HoodieInstant instant)

readCompactionPlanAsBytes

public Option<byte[]> readCompactionPlanAsBytes(HoodieInstant instant)

readIndexPlanAsBytes

public Option<byte[]> readIndexPlanAsBytes(HoodieInstant instant)

revertInstantFromInflightToRequested

public HoodieInstant revertInstantFromInflightToRequested(HoodieInstant inflightInstant)

Revert instant state from inflight to requested.

Parameters:

inflightInstant - Inflight Instant

Returns:

requested instant

revertLogCompactionInflightToRequested

public HoodieInstant revertLogCompactionInflightToRequested(HoodieInstant inflightInstant)

TODO: This method is not needed, since log compaction plan is not a immutable plan. Revert logcompaction State from inflight to requested.

Parameters:

inflightInstant - Inflight Instant

Returns:

requested instant

transitionCompactionRequestedToInflight

public HoodieInstant transitionCompactionRequestedToInflight(HoodieInstant requestedInstant)

Transition Compaction State from requested to inflight.

Parameters:

requestedInstant - Requested instant

Returns:

inflight instant

transitionLogCompactionRequestedToInflight

public HoodieInstant transitionLogCompactionRequestedToInflight(HoodieInstant requestedInstant)

Transition LogCompaction State from requested to inflight.

Parameters:

requestedInstant - Requested instant

Returns:

inflight instant

transitionCompactionInflightToComplete

public HoodieInstant transitionCompactionInflightToComplete(boolean shouldLock,
                                                            HoodieInstant inflightInstant,
                                                            Option<byte[]> data)

Transition Compaction State from inflight to Committed.

Parameters:

shouldLock - Whether to hold the lock when performing transition

inflightInstant - Inflight instant

data - Extra Metadata

Returns:

commit instant

transitionLogCompactionInflightToComplete

public HoodieInstant transitionLogCompactionInflightToComplete(boolean shouldLock,
                                                               HoodieInstant inflightInstant,
                                                               Option<byte[]> data)

Transition Log Compaction State from inflight to Committed.

Parameters:

shouldLock - Whether to hold the lock when performing transition

inflightInstant - Inflight instant

data - Extra Metadata

Returns:

commit instant

transitionCleanInflightToComplete

public HoodieInstant transitionCleanInflightToComplete(boolean shouldLock,
                                                       HoodieInstant inflightInstant,
                                                       Option<byte[]> data)

Transition Clean State from inflight to Committed.

Parameters:

shouldLock - Whether to hold the lock when performing transition

inflightInstant - Inflight instant

data - Extra Metadata

Returns:

commit instant

transitionCleanRequestedToInflight

public HoodieInstant transitionCleanRequestedToInflight(HoodieInstant requestedInstant,
                                                        Option<byte[]> data)

Transition Clean State from requested to inflight.

Parameters:

requestedInstant - requested instant

data - Optional data to be stored

Returns:

commit instant

transitionRollbackInflightToComplete

public HoodieInstant transitionRollbackInflightToComplete(boolean shouldLock,
                                                          HoodieInstant inflightInstant,
                                                          Option<byte[]> data)

Transition Rollback State from inflight to Committed.

Parameters:

shouldLock - Whether to hold the lock when performing transition

inflightInstant - Inflight instant

data - Extra Metadata

Returns:

commit instant

transitionRollbackRequestedToInflight

public HoodieInstant transitionRollbackRequestedToInflight(HoodieInstant requestedInstant)

Transition Rollback State from requested to inflight.

Parameters:

requestedInstant - requested instant

Returns:

commit instant

transitionRestoreRequestedToInflight

public HoodieInstant transitionRestoreRequestedToInflight(HoodieInstant requestedInstant)

Transition Restore State from requested to inflight.

Parameters:

requestedInstant - requested instant

Returns:

commit instant

transitionReplaceRequestedToInflight

public HoodieInstant transitionReplaceRequestedToInflight(HoodieInstant requestedInstant,
                                                          Option<byte[]> data)

Transition replace requested file to replace inflight.

Parameters:

requestedInstant - Requested instant

data - Extra Metadata

Returns:

inflight instant

transitionReplaceInflightToComplete

public HoodieInstant transitionReplaceInflightToComplete(boolean shouldLock,
                                                         HoodieInstant inflightInstant,
                                                         Option<byte[]> data)

Transition replace inflight to Committed.

Parameters:

shouldLock - Whether to hold the lock when performing transition

inflightInstant - Inflight instant

data - Extra Metadata

Returns:

commit instant

transitionStateToComplete

protected void transitionStateToComplete(boolean shouldLock,
                                         HoodieInstant fromInstant,
                                         HoodieInstant toInstant,
                                         Option<byte[]> data)

transitionPendingState

protected void transitionPendingState(HoodieInstant fromInstant,
                                      HoodieInstant toInstant,
                                      Option<byte[]> data,
                                      boolean allowRedundantTransitions)

revertCompleteToInflight

protected void revertCompleteToInflight(HoodieInstant completed,
                                        HoodieInstant inflight)

transitionRequestedToInflight

public void transitionRequestedToInflight(String commitType,
                                          String inFlightInstant)

transitionRequestedToInflight

public void transitionRequestedToInflight(HoodieInstant requested,
                                          Option<byte[]> content)

transitionRequestedToInflight

public void transitionRequestedToInflight(HoodieInstant requested,
                                          Option<byte[]> content,
                                          boolean allowRedundantTransitions)

saveToCompactionRequested

public void saveToCompactionRequested(HoodieInstant instant,
                                      Option<byte[]> content)

saveToCompactionRequested

public void saveToCompactionRequested(HoodieInstant instant,
                                      Option<byte[]> content,
                                      boolean overwrite)

saveToLogCompactionRequested

public void saveToLogCompactionRequested(HoodieInstant instant,
                                         Option<byte[]> content)

saveToLogCompactionRequested

public void saveToLogCompactionRequested(HoodieInstant instant,
                                         Option<byte[]> content,
                                         boolean overwrite)

saveToPendingReplaceCommit

public void saveToPendingReplaceCommit(HoodieInstant instant,
                                       Option<byte[]> content)

Saves content for requested REPLACE instant.

saveToCleanRequested

public void saveToCleanRequested(HoodieInstant instant,
                                 Option<byte[]> content)

saveToRollbackRequested

public void saveToRollbackRequested(HoodieInstant instant,
                                    Option<byte[]> content)

saveToRestoreRequested

public void saveToRestoreRequested(HoodieInstant instant,
                                   Option<byte[]> content)

transitionIndexRequestedToInflight

public HoodieInstant transitionIndexRequestedToInflight(HoodieInstant requestedInstant,
                                                        Option<byte[]> data)

Transition index instant state from requested to inflight.

Parameters:

requestedInstant - Inflight Instant

Returns:

inflight instant

transitionIndexInflightToComplete

public HoodieInstant transitionIndexInflightToComplete(boolean shouldLock,
                                                       HoodieInstant inflightInstant,
                                                       Option<byte[]> data)

Transition index instant state from inflight to completed.

Parameters:

shouldLock - Whether to hold the lock when performing transition

inflightInstant - Inflight Instant

Returns:

completed instant

revertIndexInflightToRequested

public HoodieInstant revertIndexInflightToRequested(HoodieInstant inflightInstant)

Revert index instant state from inflight to requested.

Parameters:

inflightInstant - Inflight Instant

Returns:

requested instant

saveToPendingIndexAction

public void saveToPendingIndexAction(HoodieInstant instant,
                                     Option<byte[]> content)

Save content for inflight/requested index instant.

createFileInMetaPath

protected void createFileInMetaPath(String filename,
                                    Option<byte[]> content,
                                    boolean allowOverwrite)

createCompleteFileInMetaPath

protected void createCompleteFileInMetaPath(boolean shouldLock,
                                            HoodieInstant instant,
                                            Option<byte[]> content)

readDataFromPath

protected Option<byte[]> readDataFromPath(StoragePath detailPath)

reload

public HoodieActiveTimeline reload()

copyInstant

public void copyInstant(HoodieInstant instant,
                        StoragePath dstDir)