org.apache.flink.batch.connectors.pulsar

Class SplitUtils

java.lang.Object

org.apache.flink.batch.connectors.pulsar.SplitUtils

public class SplitUtils
extends Object

Utilities for split generation logic.

Constructor Summary

Constructors

Constructor and Description
SplitUtils()

Method Summary

Modifier and Type	Method and Description
static Collection<InputLedger>	getLedgersInBetween(String topic, org.apache.pulsar.client.impl.MessageIdImpl start, org.apache.pulsar.client.impl.MessageIdImpl end, CachedClients cachedClients)
static List<List<InputLedger>>	partitionToNSplits(List<InputLedger> ledgers, int parallelism) Best effort partitioning based on ledger size (use backtracking method).
static long	sizeOfLedgerList(List<InputLedger> ledgers)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SplitUtils

public SplitUtils()

Method Detail

getLedgersInBetween

public static Collection<InputLedger> getLedgersInBetween(String topic,
                                                          org.apache.pulsar.client.impl.MessageIdImpl start,
                                                          org.apache.pulsar.client.impl.MessageIdImpl end,
                                                          CachedClients cachedClients)
                                                   throws Exception

Throws:

Exception

partitionToNSplits

public static List<List<InputLedger>> partitionToNSplits(List<InputLedger> ledgers,
                                                         int parallelism)

Best effort partitioning based on ledger size (use backtracking method). If ledger number less than or equal to the target parallelism, make a partition for each ledger.

Parameters:

ledgers - The ledgers list to be partitioned.

parallelism - The number of target partition.

Returns:

Ledgers grouped into partitions.

sizeOfLedgerList

public static long sizeOfLedgerList(List<InputLedger> ledgers)