org.apache.hudi.table.action.compact.strategy

Class DayBasedCompactionStrategy

java.lang.Object

org.apache.hudi.table.action.compact.strategy.CompactionStrategy

org.apache.hudi.table.action.compact.strategy.BoundedIOCompactionStrategy

org.apache.hudi.table.action.compact.strategy.DayBasedCompactionStrategy

All Implemented Interfaces:

Serializable

Direct Known Subclasses:

BoundedPartitionAwareCompactionStrategy

public class DayBasedCompactionStrategy
extends BoundedIOCompactionStrategy

This strategy orders compactions in reverse order of creation of Hive Partitions. It helps to compact data in latest partitions first and then older capped at the Total_IO allowed.

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
protected static Comparator<String>	comparator
protected static String	DATE_PARTITION_FORMAT

Fields inherited from class org.apache.hudi.table.action.compact.strategy.CompactionStrategy

TOTAL_IO_MB, TOTAL_IO_READ_MB, TOTAL_IO_WRITE_MB, TOTAL_LOG_FILE_SIZE, TOTAL_LOG_FILES

Constructor Summary

Constructors

Constructor and Description
DayBasedCompactionStrategy()

Method Summary

Modifier and Type	Method and Description
List<String>	filterPartitionPaths(HoodieWriteConfig writeConfig, List<String> allPartitionPaths) Filter the partition paths based on compaction strategy.
Comparator<String>	getComparator()
protected static String	getPartitionPathWithoutPartitionKeys(String partitionPath) If is Hive style partition path, convert it to regular partition path.

Methods inherited from class org.apache.hudi.table.action.compact.strategy.BoundedIOCompactionStrategy

orderAndFilter

Methods inherited from class org.apache.hudi.table.action.compact.strategy.CompactionStrategy

captureMetrics, generateCompactionPlan

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DATE_PARTITION_FORMAT

protected static final String DATE_PARTITION_FORMAT

See Also:

Constant Field Values

comparator

protected static Comparator<String> comparator

Constructor Detail

DayBasedCompactionStrategy

public DayBasedCompactionStrategy()

Method Detail

getComparator

public Comparator<String> getComparator()

filterPartitionPaths

public List<String> filterPartitionPaths(HoodieWriteConfig writeConfig,
                                         List<String> allPartitionPaths)

Description copied from class: CompactionStrategy

Filter the partition paths based on compaction strategy.

Overrides:

filterPartitionPaths in class CompactionStrategy

Returns:

getPartitionPathWithoutPartitionKeys

protected static String getPartitionPathWithoutPartitionKeys(String partitionPath)

If is Hive style partition path, convert it to regular partition path. e.g. year=2019/month=11/day=24 => 2019/11/24