org.apache.hudi.table.action.compact.strategy

Class UnBoundedPartitionAwareCompactionStrategy

java.lang.Object

org.apache.hudi.table.action.compact.strategy.CompactionStrategy

org.apache.hudi.table.action.compact.strategy.UnBoundedPartitionAwareCompactionStrategy

All Implemented Interfaces:

Serializable

public class UnBoundedPartitionAwareCompactionStrategy
extends CompactionStrategy

UnBoundedPartitionAwareCompactionStrategy is a custom UnBounded Strategy. This will filter all the partitions that are eligible to be compacted by a BoundedPartitionAwareCompactionStrategy and return the result. This is done so that a long running UnBoundedPartitionAwareCompactionStrategy does not step over partitions in a shorter running BoundedPartitionAwareCompactionStrategy. Essentially, this is an inverse of the partitions chosen in BoundedPartitionAwareCompactionStrategy

See Also:

CompactionStrategy, Serialized Form

Field Summary

Fields inherited from class org.apache.hudi.table.action.compact.strategy.CompactionStrategy

TOTAL_IO_MB, TOTAL_IO_READ_MB, TOTAL_IO_WRITE_MB, TOTAL_LOG_FILE_SIZE, TOTAL_LOG_FILES

Constructor Summary

Constructors

Constructor and Description
UnBoundedPartitionAwareCompactionStrategy()

Method Summary

Modifier and Type	Method and Description
List<String>	filterPartitionPaths(HoodieWriteConfig writeConfig, List<String> partitionPaths) Filter the partition paths based on compaction strategy.
List<HoodieCompactionOperation>	orderAndFilter(HoodieWriteConfig config, List<HoodieCompactionOperation> operations, List<HoodieCompactionPlan> pendingCompactionWorkloads) Order and Filter the list of compactions.

Methods inherited from class org.apache.hudi.table.action.compact.strategy.CompactionStrategy

captureMetrics, generateCompactionPlan

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

UnBoundedPartitionAwareCompactionStrategy

public UnBoundedPartitionAwareCompactionStrategy()

Method Detail

orderAndFilter

public List<HoodieCompactionOperation> orderAndFilter(HoodieWriteConfig config,
                                                      List<HoodieCompactionOperation> operations,
                                                      List<HoodieCompactionPlan> pendingCompactionWorkloads)

Description copied from class: CompactionStrategy

Order and Filter the list of compactions. Use the metrics captured with the captureMetrics to order and filter out compactions

Overrides:

orderAndFilter in class CompactionStrategy

Parameters:

config - config for this compaction is passed in

operations - list of compactions collected

pendingCompactionWorkloads - Pending Compaction Plans for strategy to schedule next compaction plan

Returns:

list of compactions to perform in this run

filterPartitionPaths

public List<String> filterPartitionPaths(HoodieWriteConfig writeConfig,
                                         List<String> partitionPaths)

Description copied from class: CompactionStrategy

Filter the partition paths based on compaction strategy.

Overrides:

filterPartitionPaths in class CompactionStrategy

Returns: