org.apache.hudi.client

Class CompactionAdminClient

java.lang.Object

org.apache.hudi.client.BaseHoodieClient

org.apache.hudi.client.CompactionAdminClient

All Implemented Interfaces:

Serializable, AutoCloseable

public class CompactionAdminClient
extends BaseHoodieClient

Client to perform admin operations related to compaction.

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	CompactionAdminClient.CompactionValidationException
static class	CompactionAdminClient.RenameInfo
static class	CompactionAdminClient.RenameOpResult Holds Operation result for Renaming.
static class	CompactionAdminClient.ValidationOpResult Holds Operation result for Renaming.

Field Summary

Fields inherited from class org.apache.hudi.client.BaseHoodieClient

basePath, config, context, heartbeatClient, metrics, storage, storageConf, txnManager

Constructor Summary

Constructors

Constructor and Description
CompactionAdminClient(HoodieEngineContext context, String basePath)

Method Summary

Modifier and Type	Method and Description
protected static void	renameLogFile(HoodieTableMetaClient metaClient, HoodieLogFile oldLogFile, HoodieLogFile newLogFile) Rename log files.
List<CompactionAdminClient.RenameOpResult>	repairCompaction(String compactionInstant, int parallelism, boolean dryRun) Renames delta files to make file-slices consistent with the timeline as dictated by Hoodie metadata.
List<CompactionAdminClient.RenameOpResult>	unscheduleCompactionFileId(HoodieFileGroupId fgId, boolean skipValidation, boolean dryRun) Remove a fileId from pending compaction.
List<CompactionAdminClient.RenameOpResult>	unscheduleCompactionPlan(String compactionInstant, boolean skipValidation, int parallelism, boolean dryRun) Un-schedules compaction plan.
List<CompactionAdminClient.ValidationOpResult>	validateCompactionPlan(HoodieTableMetaClient metaClient, String compactionInstant, int parallelism) Validate all compaction operations in a compaction plan.

Methods inherited from class org.apache.hudi.client.BaseHoodieClient

close, createMetaClient, createNewInstantTime, createNewInstantTime, finalizeWrite, getConfig, getEngineContext, getHeartbeatClient, getTimelineServer, initWrapperFSMetrics, resolveWriteConflict, writeTableMetadata

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CompactionAdminClient

public CompactionAdminClient(HoodieEngineContext context,
                             String basePath)

Method Detail

validateCompactionPlan

public List<CompactionAdminClient.ValidationOpResult> validateCompactionPlan(HoodieTableMetaClient metaClient,
                                                                             String compactionInstant,
                                                                             int parallelism)
                                                                      throws IOException

Validate all compaction operations in a compaction plan. Verifies the file-slices are consistent with corresponding compaction operations.

Parameters:

metaClient - Hoodie Table Meta Client

compactionInstant - Compaction Instant

Throws:

IOException

unscheduleCompactionPlan

public List<CompactionAdminClient.RenameOpResult> unscheduleCompactionPlan(String compactionInstant,
                                                                           boolean skipValidation,
                                                                           int parallelism,
                                                                           boolean dryRun)
                                                                    throws Exception

Un-schedules compaction plan. Remove All compaction operation scheduled.

Parameters:

compactionInstant - Compaction Instant

skipValidation - Skip validation step

parallelism - Parallelism

dryRun - Dry Run

Throws:

Exception

unscheduleCompactionFileId

public List<CompactionAdminClient.RenameOpResult> unscheduleCompactionFileId(HoodieFileGroupId fgId,
                                                                             boolean skipValidation,
                                                                             boolean dryRun)
                                                                      throws Exception

Remove a fileId from pending compaction. Removes the associated compaction operation and rename delta-files that were generated for that file-id after the compaction operation was scheduled. This operation MUST be executed with compactions and writer turned OFF.

Parameters:

fgId - FileGroupId to be unscheduled

skipValidation - Skip validation

dryRun - Dry Run Mode

Throws:

Exception

repairCompaction

public List<CompactionAdminClient.RenameOpResult> repairCompaction(String compactionInstant,
                                                                   int parallelism,
                                                                   boolean dryRun)
                                                            throws Exception

Renames delta files to make file-slices consistent with the timeline as dictated by Hoodie metadata. Use when compaction unschedule fails partially. This operation MUST be executed with compactions and writer turned OFF.

Parameters:

compactionInstant - Compaction Instant to be repaired

dryRun - Dry Run Mode

Throws:

Exception

renameLogFile

protected static void renameLogFile(HoodieTableMetaClient metaClient,
                                    HoodieLogFile oldLogFile,
                                    HoodieLogFile newLogFile)
                             throws IOException

Rename log files. This is done for un-scheduling a pending compaction operation NOTE: Can only be used safely when no writer (ingestion/compaction) is running.

Parameters:

metaClient - Hoodie Table Meta-Client

oldLogFile - Old Log File

newLogFile - New Log File

Throws:

IOException