org.apache.flink.connector.pulsar.source.reader.source

Class PulsarUnorderedSourceReader<OUT>

java.lang.Object

org.apache.flink.connector.base.source.reader.SourceReaderBase<org.apache.pulsar.client.api.Message<byte[]>,OUT,PulsarPartitionSplit,PulsarPartitionSplitState>

org.apache.flink.connector.pulsar.source.reader.source.PulsarUnorderedSourceReader<OUT>

All Implemented Interfaces:

AutoCloseable, org.apache.flink.api.common.state.CheckpointListener, org.apache.flink.api.connector.source.SourceReader<OUT,PulsarPartitionSplit>

@Internal
public class PulsarUnorderedSourceReader<OUT>
extends org.apache.flink.connector.base.source.reader.SourceReaderBase<org.apache.pulsar.client.api.Message<byte[]>,OUT,PulsarPartitionSplit,PulsarPartitionSplitState>

The source reader for pulsar subscription Shared and Key_Shared, which consumes the unordered messages.

Field Summary

Fields

Modifier and Type	Field and Description
protected org.apache.pulsar.client.admin.PulsarAdmin	pulsarAdmin
protected org.apache.pulsar.client.api.PulsarClient	pulsarClient
protected SourceConfiguration	sourceConfiguration

Fields inherited from class org.apache.flink.connector.base.source.reader.SourceReaderBase

config, context, options, recordEmitter, splitFetcherManager

Constructor Summary

Constructors

Constructor and Description

PulsarUnorderedSourceReader(org.apache.flink.connector.base.source.reader.synchronization.FutureCompletingBlockingQueue<org.apache.flink.connector.base.source.reader.RecordsWithSplitIds<org.apache.pulsar.client.api.Message<byte[]>>> elementsQueue, java.util.function.Supplier<PulsarUnorderedPartitionSplitReader> splitReaderSupplier, PulsarRecordEmitter<OUT> recordEmitter, org.apache.flink.api.connector.source.SourceReaderContext context, SourceConfiguration sourceConfiguration, org.apache.pulsar.client.api.PulsarClient pulsarClient, org.apache.pulsar.client.admin.PulsarAdmin pulsarAdmin, org.apache.pulsar.client.api.transaction.TransactionCoordinatorClient coordinatorClient)

Method Summary

Modifier and Type	Method and Description
void	close()
protected void	closeFinishedSplits(Set<String> finishedSplitIds)
protected PulsarPartitionSplitState	initializedState(PulsarPartitionSplit split)
void	notifyCheckpointComplete(long checkpointId)
protected void	onSplitFinished(Map<String,PulsarPartitionSplitState> finishedSplitIds)
List<PulsarPartitionSplit>	snapshotState(long checkpointId)
protected PulsarPartitionSplit	toSplitType(String splitId, PulsarPartitionSplitState splitState)

Methods inherited from class org.apache.flink.connector.base.source.reader.SourceReaderBase

addSplits, getNumberOfCurrentlyAssignedSplits, handleSourceEvents, isAvailable, notifyNoMoreSplits, pollNext, start

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.api.common.state.CheckpointListener

notifyCheckpointAborted

Field Detail

sourceConfiguration

protected final SourceConfiguration sourceConfiguration

pulsarClient

protected final org.apache.pulsar.client.api.PulsarClient pulsarClient

pulsarAdmin

protected final org.apache.pulsar.client.admin.PulsarAdmin pulsarAdmin

Constructor Detail

PulsarUnorderedSourceReader

public PulsarUnorderedSourceReader(org.apache.flink.connector.base.source.reader.synchronization.FutureCompletingBlockingQueue<org.apache.flink.connector.base.source.reader.RecordsWithSplitIds<org.apache.pulsar.client.api.Message<byte[]>>> elementsQueue,
                                   java.util.function.Supplier<PulsarUnorderedPartitionSplitReader> splitReaderSupplier,
                                   PulsarRecordEmitter<OUT> recordEmitter,
                                   org.apache.flink.api.connector.source.SourceReaderContext context,
                                   SourceConfiguration sourceConfiguration,
                                   org.apache.pulsar.client.api.PulsarClient pulsarClient,
                                   org.apache.pulsar.client.admin.PulsarAdmin pulsarAdmin,
                                   @Nullable
                                   org.apache.pulsar.client.api.transaction.TransactionCoordinatorClient coordinatorClient)

Method Detail

onSplitFinished

protected void onSplitFinished(Map<String,PulsarPartitionSplitState> finishedSplitIds)

Specified by:

onSplitFinished in class org.apache.flink.connector.base.source.reader.SourceReaderBase<org.apache.pulsar.client.api.Message<byte[]>,OUT,PulsarPartitionSplit,PulsarPartitionSplitState>

snapshotState

public List<PulsarPartitionSplit> snapshotState(long checkpointId)

Specified by:

snapshotState in interface org.apache.flink.api.connector.source.SourceReader<OUT,PulsarPartitionSplit>

Overrides:

snapshotState in class org.apache.flink.connector.base.source.reader.SourceReaderBase<org.apache.pulsar.client.api.Message<byte[]>,OUT,PulsarPartitionSplit,PulsarPartitionSplitState>

notifyCheckpointComplete

public void notifyCheckpointComplete(long checkpointId)
                              throws Exception

Throws:

Exception

close

public void close()
           throws Exception

Specified by:

close in interface AutoCloseable

Throws:

Exception

initializedState

protected PulsarPartitionSplitState initializedState(PulsarPartitionSplit split)

Specified by:

initializedState in class org.apache.flink.connector.base.source.reader.SourceReaderBase<org.apache.pulsar.client.api.Message<byte[]>,OUT,PulsarPartitionSplit,PulsarPartitionSplitState>

toSplitType

protected PulsarPartitionSplit toSplitType(String splitId,
                                           PulsarPartitionSplitState splitState)

Specified by:

toSplitType in class org.apache.flink.connector.base.source.reader.SourceReaderBase<org.apache.pulsar.client.api.Message<byte[]>,OUT,PulsarPartitionSplit,PulsarPartitionSplitState>

closeFinishedSplits

protected void closeFinishedSplits(Set<String> finishedSplitIds)