org.apache.flink.connector.pulsar.source.reader.split

Class PulsarOrderedPartitionSplitReader

java.lang.Object

org.apache.flink.connector.pulsar.source.reader.split.PulsarOrderedPartitionSplitReader

All Implemented Interfaces:

org.apache.flink.connector.base.source.reader.splitreader.SplitReader<org.apache.pulsar.client.api.Message<byte[]>,PulsarPartitionSplit>

@Internal
public class PulsarOrderedPartitionSplitReader
extends Object

The split reader a given PulsarPartitionSplit, it would be closed once the PulsarOrderedSourceReader is closed.

Field Summary

Fields

Modifier and Type	Field and Description
protected org.apache.pulsar.client.api.CryptoKeyReader	cryptoKeyReader
protected org.apache.pulsar.client.admin.PulsarAdmin	pulsarAdmin
protected org.apache.pulsar.client.api.PulsarClient	pulsarClient
protected org.apache.pulsar.client.api.Consumer<byte[]>	pulsarConsumer
protected PulsarPartitionSplit	registeredSplit
protected org.apache.pulsar.client.api.Schema<byte[]>	schema
protected SourceConfiguration	sourceConfiguration
protected AtomicBoolean	wakeup

Constructor Summary

Constructors

Constructor and Description
PulsarOrderedPartitionSplitReader(org.apache.pulsar.client.api.PulsarClient pulsarClient, org.apache.pulsar.client.admin.PulsarAdmin pulsarAdmin, SourceConfiguration sourceConfiguration, org.apache.pulsar.client.api.Schema<byte[]> schema, org.apache.pulsar.client.api.CryptoKeyReader cryptoKeyReader)

Method Summary

Modifier and Type	Method and Description
void	close()
protected org.apache.pulsar.client.api.Consumer<byte[]>	createPulsarConsumer(PulsarPartitionSplit split) Create a specified Consumer by the given split information.
protected org.apache.pulsar.client.api.Consumer<byte[]>	createPulsarConsumer(TopicPartition partition) Create a specified Consumer by the given topic partition.
org.apache.flink.connector.base.source.reader.RecordsWithSplitIds<org.apache.pulsar.client.api.Message<byte[]>>	fetch()
protected void	finishedPollMessage(org.apache.pulsar.client.api.Message<?> message)
void	handleSplitsChanges(org.apache.flink.connector.base.source.reader.splitreader.SplitsChange<PulsarPartitionSplit> splitsChanges)
protected boolean	isNotWakeup()
void	notifyCheckpointComplete(TopicPartition partition, org.apache.pulsar.client.api.MessageId offsetsToCommit)
protected org.apache.pulsar.client.api.Message<byte[]>	pollMessage(java.time.Duration timeout)
protected void	startConsumer(PulsarPartitionSplit split, org.apache.pulsar.client.api.Consumer<?> consumer)
void	wakeUp()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

pulsarClient

protected final org.apache.pulsar.client.api.PulsarClient pulsarClient

pulsarAdmin

protected final org.apache.pulsar.client.admin.PulsarAdmin pulsarAdmin

sourceConfiguration

protected final SourceConfiguration sourceConfiguration

schema

protected final org.apache.pulsar.client.api.Schema<byte[]> schema

cryptoKeyReader

@Nullable
protected final org.apache.pulsar.client.api.CryptoKeyReader cryptoKeyReader

wakeup

protected final AtomicBoolean wakeup

pulsarConsumer

protected org.apache.pulsar.client.api.Consumer<byte[]> pulsarConsumer

registeredSplit

protected PulsarPartitionSplit registeredSplit

Constructor Detail

PulsarOrderedPartitionSplitReader

public PulsarOrderedPartitionSplitReader(org.apache.pulsar.client.api.PulsarClient pulsarClient,
                                         org.apache.pulsar.client.admin.PulsarAdmin pulsarAdmin,
                                         SourceConfiguration sourceConfiguration,
                                         org.apache.pulsar.client.api.Schema<byte[]> schema,
                                         @Nullable
                                         org.apache.pulsar.client.api.CryptoKeyReader cryptoKeyReader)

Method Detail

pollMessage

protected org.apache.pulsar.client.api.Message<byte[]> pollMessage(java.time.Duration timeout)
                                                            throws org.apache.pulsar.client.api.PulsarClientException

Throws:

org.apache.pulsar.client.api.PulsarClientException

finishedPollMessage

protected void finishedPollMessage(org.apache.pulsar.client.api.Message<?> message)

startConsumer

protected void startConsumer(PulsarPartitionSplit split,
                             org.apache.pulsar.client.api.Consumer<?> consumer)

notifyCheckpointComplete

public void notifyCheckpointComplete(TopicPartition partition,
                                     org.apache.pulsar.client.api.MessageId offsetsToCommit)

fetch

public org.apache.flink.connector.base.source.reader.RecordsWithSplitIds<org.apache.pulsar.client.api.Message<byte[]>> fetch()
                                                                                                                      throws IOException

Specified by:

fetch in interface org.apache.flink.connector.base.source.reader.splitreader.SplitReader<org.apache.pulsar.client.api.Message<byte[]>,PulsarPartitionSplit>

Throws:

IOException

handleSplitsChanges

public void handleSplitsChanges(org.apache.flink.connector.base.source.reader.splitreader.SplitsChange<PulsarPartitionSplit> splitsChanges)

Specified by:

handleSplitsChanges in interface org.apache.flink.connector.base.source.reader.splitreader.SplitReader<org.apache.pulsar.client.api.Message<byte[]>,PulsarPartitionSplit>

wakeUp

public void wakeUp()

Specified by:

wakeUp in interface org.apache.flink.connector.base.source.reader.splitreader.SplitReader<org.apache.pulsar.client.api.Message<byte[]>,PulsarPartitionSplit>

close

public void close()

Specified by:

close in interface org.apache.flink.connector.base.source.reader.splitreader.SplitReader<org.apache.pulsar.client.api.Message<byte[]>,PulsarPartitionSplit>

isNotWakeup

protected boolean isNotWakeup()

createPulsarConsumer

protected org.apache.pulsar.client.api.Consumer<byte[]> createPulsarConsumer(PulsarPartitionSplit split)

Create a specified Consumer by the given split information.

createPulsarConsumer

protected org.apache.pulsar.client.api.Consumer<byte[]> createPulsarConsumer(TopicPartition partition)

Create a specified Consumer by the given topic partition.