org.apache.flink.connector.pulsar.source.reader.fetcher

Class PulsarFetcherManagerBase

java.lang.Object

org.apache.flink.connector.base.source.reader.fetcher.SplitFetcherManager<org.apache.pulsar.client.api.Message<byte[]>,PulsarPartitionSplit>

org.apache.flink.connector.pulsar.source.reader.fetcher.PulsarFetcherManagerBase

Direct Known Subclasses:

PulsarOrderedFetcherManager, PulsarUnorderedFetcherManager

@Internal
public abstract class PulsarFetcherManagerBase
extends org.apache.flink.connector.base.source.reader.fetcher.SplitFetcherManager<org.apache.pulsar.client.api.Message<byte[]>,PulsarPartitionSplit>

Common fetcher manager abstraction for both ordered & unordered message.

Field Summary

Fields inherited from class org.apache.flink.connector.base.source.reader.fetcher.SplitFetcherManager

fetchers

Constructor Summary

Constructors

Modifier

Constructor and Description

protected

PulsarFetcherManagerBase(org.apache.flink.connector.base.source.reader.synchronization.FutureCompletingBlockingQueue<org.apache.flink.connector.base.source.reader.RecordsWithSplitIds<org.apache.pulsar.client.api.Message<byte[]>>> elementsQueue, java.util.function.Supplier<org.apache.flink.connector.base.source.reader.splitreader.SplitReader<org.apache.pulsar.client.api.Message<byte[]>,PulsarPartitionSplit>> splitReaderSupplier) Creates a new SplitFetcherManager with multiple I/O threads.

Method Summary

Modifier and Type	Method and Description
void	addSplits(List<PulsarPartitionSplit> splitsToAdd) Override this method for supporting multiple thread fetching, one fetcher thread for one split.
void	closeFetcher(String splitId) Close the finished split related fetcher.
protected org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher<org.apache.pulsar.client.api.Message<byte[]>,PulsarPartitionSplit>	getOrCreateFetcher(String splitId)
protected void	startFetcher(org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher<org.apache.pulsar.client.api.Message<byte[]>,PulsarPartitionSplit> fetcher)

Methods inherited from class org.apache.flink.connector.base.source.reader.fetcher.SplitFetcherManager

checkErrors, close, createSplitFetcher, getNumAliveFetchers, maybeShutdownFinishedFetchers

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

PulsarFetcherManagerBase

protected PulsarFetcherManagerBase(org.apache.flink.connector.base.source.reader.synchronization.FutureCompletingBlockingQueue<org.apache.flink.connector.base.source.reader.RecordsWithSplitIds<org.apache.pulsar.client.api.Message<byte[]>>> elementsQueue,
                                   java.util.function.Supplier<org.apache.flink.connector.base.source.reader.splitreader.SplitReader<org.apache.pulsar.client.api.Message<byte[]>,PulsarPartitionSplit>> splitReaderSupplier)

Creates a new SplitFetcherManager with multiple I/O threads.

Parameters:

elementsQueue - The queue that is used to hand over data from the I/O thread (the fetchers) to the reader (which emits the records and book-keeps the state. This must be the same queue instance that is also passed to the SourceReaderBase.

splitReaderSupplier - The factory for the split reader that connects to the source

Method Detail

addSplits

public void addSplits(List<PulsarPartitionSplit> splitsToAdd)

Override this method for supporting multiple thread fetching, one fetcher thread for one split.

Specified by:

addSplits in class org.apache.flink.connector.base.source.reader.fetcher.SplitFetcherManager<org.apache.pulsar.client.api.Message<byte[]>,PulsarPartitionSplit>

closeFetcher

public void closeFetcher(String splitId)

Close the finished split related fetcher.

startFetcher

protected void startFetcher(org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher<org.apache.pulsar.client.api.Message<byte[]>,PulsarPartitionSplit> fetcher)

Overrides:

startFetcher in class org.apache.flink.connector.base.source.reader.fetcher.SplitFetcherManager<org.apache.pulsar.client.api.Message<byte[]>,PulsarPartitionSplit>

getOrCreateFetcher

protected org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher<org.apache.pulsar.client.api.Message<byte[]>,PulsarPartitionSplit> getOrCreateFetcher(String splitId)