org.apache.hudi.execution.bulkinsert

Enum BulkInsertSortMode

java.lang.Object

java.lang.Enum<BulkInsertSortMode>

org.apache.hudi.execution.bulkinsert.BulkInsertSortMode

All Implemented Interfaces:

Serializable, Comparable<BulkInsertSortMode>

@EnumDescription(value="Modes for sorting records during bulk insert.")
public enum BulkInsertSortMode
extends Enum<BulkInsertSortMode>

Bulk insert sort mode.

Enum Constant Summary

Enum Constants

Enum Constant and Description
GLOBAL_SORT
NONE
PARTITION_PATH_REPARTITION
PARTITION_PATH_REPARTITION_AND_SORT
PARTITION_SORT

Method Summary

Modifier and Type	Method and Description
static BulkInsertSortMode	valueOf(String name) Returns the enum constant of this type with the specified name.
static BulkInsertSortMode[]	values() Returns an array containing the constants of this enum type, in the order they are declared.

Methods inherited from class java.lang.Enum

clone, compareTo, equals, finalize, getDeclaringClass, hashCode, name, ordinal, toString, valueOf

Methods inherited from class java.lang.Object

getClass, notify, notifyAll, wait, wait, wait

Enum Constant Detail

NONE

@EnumFieldDescription(value="No sorting. Fastest and matches `spark.write.parquet()` in number of files and overhead.")
public static final BulkInsertSortMode NONE

GLOBAL_SORT

@EnumFieldDescription(value="This ensures best file sizes, with lowest memory overhead at cost of sorting.")
public static final BulkInsertSortMode GLOBAL_SORT

PARTITION_SORT

@EnumFieldDescription(value="Strikes a balance by only sorting within a Spark RDD partition, still keeping the memory overhead of writing low. File sizing is not as good as GLOBAL_SORT.")
public static final BulkInsertSortMode PARTITION_SORT

PARTITION_PATH_REPARTITION

@EnumFieldDescription(value="This ensures that the data for a single physical partition in the table is written by the same Spark executor. This should only be used when input data is evenly distributed across different partition paths. If data is skewed (most records are intended for a handful of partition paths among all) then this can cause an imbalance among Spark executors.")
public static final BulkInsertSortMode PARTITION_PATH_REPARTITION

PARTITION_PATH_REPARTITION_AND_SORT

@EnumFieldDescription(value="This ensures that the data for a single physical partition in the table is written by the same Spark executor. This should only be used when input data is evenly distributed across different partition paths. Compared to PARTITION_PATH_REPARTITION, this sort mode does an additional step of sorting the records based on the partition path within a single Spark partition, given that data for multiple physical partitions can be sent to the same Spark partition and executor. If data is skewed (most records are intended for a handful of partition paths among all) then this can cause an imbalance among Spark executors.")
public static final BulkInsertSortMode PARTITION_PATH_REPARTITION_AND_SORT

Method Detail

values

public static BulkInsertSortMode[] values()

Returns an array containing the constants of this enum type, in the order they are declared. This method may be used to iterate over the constants as follows:

for (BulkInsertSortMode c : BulkInsertSortMode.values())
    System.out.println(c);

Returns:

an array containing the constants of this enum type, in the order they are declared

valueOf

public static BulkInsertSortMode valueOf(String name)

Returns the enum constant of this type with the specified name. The string must match exactly an identifier used to declare an enum constant in this type. (Extraneous whitespace characters are not permitted.)

Parameters:

name - the name of the enum constant to be returned.

Returns:

the enum constant with the specified name

Throws:

IllegalArgumentException - if this enum type has no constant with the specified name

NullPointerException - if the argument is null