org.apache.hudi.table.action.compact

Class HoodieCompactor<T,I,K,O>

java.lang.Object

org.apache.hudi.table.action.compact.HoodieCompactor<T,I,K,O>

All Implemented Interfaces:

Serializable

public abstract class HoodieCompactor<T,I,K,O>
extends Object
implements Serializable

A HoodieCompactor runs compaction on a hoodie table.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
HoodieCompactor()

Method Summary

Modifier and Type	Method and Description
List<WriteStatus>	compact(HoodieCompactionHandler compactionHandler, HoodieTableMetaClient metaClient, CompactionOperation operation, String instantTime, Option<EngineBroadcastManager> broadcastManagerOpt) Execute a single compaction operation and report back status.
List<WriteStatus>	compact(HoodieCompactionHandler compactionHandler, HoodieTableMetaClient metaClient, HoodieWriteConfig config, CompactionOperation operation, String instantTime, String maxInstantTime, Option<InstantRange> instantRange, TaskContextSupplier taskContextSupplier, CompactionExecutionHelper executionHelper) Execute a single compaction operation and report back status.
List<WriteStatus>	compact(HoodieCompactionHandler compactionHandler, HoodieTableMetaClient metaClient, HoodieWriteConfig config, CompactionOperation operation, String instantTime, String maxInstantTime, TaskContextSupplier taskContextSupplier) Execute a single compaction operation using file group reader and report back status.
HoodieData<WriteStatus>	compact(HoodieEngineContext context, WriteOperationType operationType, HoodieCompactionPlan compactionPlan, HoodieTable table, HoodieWriteConfig config, String compactionInstantTime, HoodieCompactionHandler compactionHandler) Execute compaction operations and report back status.
CompactionExecutionHelper	getCompactionExecutionStrategy(HoodieCompactionPlan compactionPlan)
Option<EngineBroadcastManager>	getEngineBroadcastManager(HoodieEngineContext context)
String	getMaxInstantTime(HoodieTableMetaClient metaClient)
abstract void	maybePersist(HoodieData<WriteStatus> writeStatus, HoodieEngineContext context, HoodieWriteConfig config, String instantTime) Maybe persist write status.
abstract void	preCompact(HoodieTable table, HoodieTimeline pendingCompactionTimeline, WriteOperationType operationType, String instantTime) Handles the compaction timeline based on the compaction instant before actual compaction.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HoodieCompactor

public HoodieCompactor()

Method Detail

preCompact

public abstract void preCompact(HoodieTable table,
                                HoodieTimeline pendingCompactionTimeline,
                                WriteOperationType operationType,
                                String instantTime)

Handles the compaction timeline based on the compaction instant before actual compaction.

Parameters:

table - HoodieTable instance to use.

pendingCompactionTimeline - pending compaction timeline.

instantTime - compaction instant

maybePersist

public abstract void maybePersist(HoodieData<WriteStatus> writeStatus,
                                  HoodieEngineContext context,
                                  HoodieWriteConfig config,
                                  String instantTime)

Maybe persist write status.

Parameters:

writeStatus - HoodieData of WriteStatus.

getEngineBroadcastManager

public Option<EngineBroadcastManager> getEngineBroadcastManager(HoodieEngineContext context)

Parameters:

context - HoodieEngineContext instance

Returns:

the EngineBroadcastManager if available.

compact

public HoodieData<WriteStatus> compact(HoodieEngineContext context,
                                       WriteOperationType operationType,
                                       HoodieCompactionPlan compactionPlan,
                                       HoodieTable table,
                                       HoodieWriteConfig config,
                                       String compactionInstantTime,
                                       HoodieCompactionHandler compactionHandler)

Execute compaction operations and report back status.

compact

public List<WriteStatus> compact(HoodieCompactionHandler compactionHandler,
                                 HoodieTableMetaClient metaClient,
                                 HoodieWriteConfig config,
                                 CompactionOperation operation,
                                 String instantTime,
                                 String maxInstantTime,
                                 TaskContextSupplier taskContextSupplier)
                          throws IOException

Execute a single compaction operation using file group reader and report back status.

Throws:

IOException

compact

public List<WriteStatus> compact(HoodieCompactionHandler compactionHandler,
                                 HoodieTableMetaClient metaClient,
                                 HoodieWriteConfig config,
                                 CompactionOperation operation,
                                 String instantTime,
                                 String maxInstantTime,
                                 Option<InstantRange> instantRange,
                                 TaskContextSupplier taskContextSupplier,
                                 CompactionExecutionHelper executionHelper)
                          throws IOException

Execute a single compaction operation and report back status.

Throws:

IOException

compact

public List<WriteStatus> compact(HoodieCompactionHandler compactionHandler,
                                 HoodieTableMetaClient metaClient,
                                 CompactionOperation operation,
                                 String instantTime,
                                 Option<EngineBroadcastManager> broadcastManagerOpt)
                          throws IOException

Execute a single compaction operation and report back status.

Throws:

IOException

getMaxInstantTime

public String getMaxInstantTime(HoodieTableMetaClient metaClient)

getCompactionExecutionStrategy

public CompactionExecutionHelper getCompactionExecutionStrategy(HoodieCompactionPlan compactionPlan)