org.apache.hudi.table.action.compact.strategy

Class LogFileSizeBasedCompactionStrategy

java.lang.Object

org.apache.hudi.table.action.compact.strategy.CompactionStrategy

org.apache.hudi.table.action.compact.strategy.BoundedIOCompactionStrategy

org.apache.hudi.table.action.compact.strategy.LogFileSizeBasedCompactionStrategy

All Implemented Interfaces:

Serializable, Comparator<HoodieCompactionOperation>

public class LogFileSizeBasedCompactionStrategy
extends BoundedIOCompactionStrategy
implements Comparator<HoodieCompactionOperation>

LogFileSizeBasedCompactionStrategy orders the compactions based on the total log files size, filters the file group which log files size is greater than the threshold and limits the compactions within a configured IO bound.

See Also:

BoundedIOCompactionStrategy, CompactionStrategy, Serialized Form

Field Summary

Fields inherited from class org.apache.hudi.table.action.compact.strategy.CompactionStrategy

TOTAL_IO_MB, TOTAL_IO_READ_MB, TOTAL_IO_WRITE_MB, TOTAL_LOG_FILE_SIZE, TOTAL_LOG_FILES

Constructor Summary

Constructors

Constructor and Description
LogFileSizeBasedCompactionStrategy()

Method Summary

Modifier and Type	Method and Description
int	compare(HoodieCompactionOperation op1, HoodieCompactionOperation op2)
List<HoodieCompactionOperation>	orderAndFilter(HoodieWriteConfig writeConfig, List<HoodieCompactionOperation> operations, List<HoodieCompactionPlan> pendingCompactionPlans) Order and Filter the list of compactions.

Methods inherited from class org.apache.hudi.table.action.compact.strategy.CompactionStrategy

captureMetrics, filterPartitionPaths, generateCompactionPlan

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.util.Comparator

comparing, comparing, comparingDouble, comparingInt, comparingLong, equals, naturalOrder, nullsFirst, nullsLast, reversed, reverseOrder, thenComparing, thenComparing, thenComparing, thenComparingDouble, thenComparingInt, thenComparingLong

Constructor Detail

LogFileSizeBasedCompactionStrategy

public LogFileSizeBasedCompactionStrategy()

Method Detail

orderAndFilter

public List<HoodieCompactionOperation> orderAndFilter(HoodieWriteConfig writeConfig,
                                                      List<HoodieCompactionOperation> operations,
                                                      List<HoodieCompactionPlan> pendingCompactionPlans)

Description copied from class: CompactionStrategy

Order and Filter the list of compactions. Use the metrics captured with the captureMetrics to order and filter out compactions

Overrides:

orderAndFilter in class BoundedIOCompactionStrategy

Parameters:

writeConfig - config for this compaction is passed in

operations - list of compactions collected

pendingCompactionPlans - Pending Compaction Plans for strategy to schedule next compaction plan

Returns:

list of compactions to perform in this run

compare

public int compare(HoodieCompactionOperation op1,
                   HoodieCompactionOperation op2)

Specified by:

compare in interface Comparator<HoodieCompactionOperation>