org.apache.hudi.table

Interface BulkInsertPartitioner<I>

public interface BulkInsertPartitioner<I>

Repartition input records into at least expected number of output spark partitions. It should give below guarantees - Output spark partition will have records from only one hoodie partition. - Average records per output spark partitions should be almost equal to (#inputRecords / #outputSparkPartitions) to avoid possible skews.

Method Summary

Modifier and Type	Method and Description
boolean	arePartitionRecordsSorted()
I	repartitionRecords(I records, int outputSparkPartitions) Repartitions the input records into at least expected number of output spark partitions.

Method Detail

repartitionRecords

I repartitionRecords(I records,
                     int outputSparkPartitions)

Repartitions the input records into at least expected number of output spark partitions.

Parameters:

records - Input Hoodie records

outputSparkPartitions - Expected number of output partitions

Returns:

arePartitionRecordsSorted

boolean arePartitionRecordsSorted()

Returns:

true if the records within a partition are sorted; false otherwise.