com.datastax.oss.dsbulk.partitioner

Class TokenRangeClusterer

java.lang.Object

com.datastax.oss.dsbulk.partitioner.TokenRangeClusterer

public class TokenRangeClusterer
extends Object

Groups small, contiguous token ranges sharing the same replicas in order to reduce the total number of splits.

Useful mostly with virtual nodes, which may create lots of small token range splits.

Constructor Summary

Constructors

Constructor and Description
TokenRangeClusterer(BulkTokenFactory tokenFactory)

Method Summary

Modifier and Type	Method and Description
List<BulkTokenRange>	group(List<BulkTokenRange> ranges, int groupCount, int maxGroupSize) Groups contiguous ranges together as long as they are contiguous and share the same replicas.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TokenRangeClusterer

public TokenRangeClusterer(@NonNull
                           BulkTokenFactory tokenFactory)

Method Detail

group

@NonNull
public List<BulkTokenRange> group(List<BulkTokenRange> ranges,
                                           int groupCount,
                                           int maxGroupSize)

Groups contiguous ranges together as long as they are contiguous and share the same replicas.

Note: the grouping algorithm used in DSBulk is different from the one used by the Spark connector: the former favors groups of contiguous ranges sharing the same replicas (in order to make it possible to route a range read to a coordinator that is also a replica), whereas the latter favors data locality (i.e. groups even non-continguous ranges as long as they share at least one common replica).