org.apache.flink.connector.pulsar.source.reader.split

Class PulsarOrderedPartitionSplitReader<OUT>

java.lang.Object

org.apache.flink.connector.pulsar.source.reader.split.PulsarOrderedPartitionSplitReader<OUT>

Type Parameters:

OUT - the type of the pulsar source message that would be serialized to downstream.

All Implemented Interfaces:

org.apache.flink.connector.base.source.reader.splitreader.SplitReader<PulsarMessage<OUT>,PulsarPartitionSplit>

@Internal
public class PulsarOrderedPartitionSplitReader<OUT>
extends Object

The split reader a given PulsarPartitionSplit, it would be closed once the PulsarOrderedSourceReader is closed.

Field Summary

Fields

Modifier and Type	Field and Description
protected PulsarDeserializationSchema<OUT>	deserializationSchema
protected org.apache.pulsar.client.admin.PulsarAdmin	pulsarAdmin
protected org.apache.pulsar.client.api.PulsarClient	pulsarClient
protected org.apache.pulsar.client.api.Consumer<?>	pulsarConsumer
protected PulsarPartitionSplit	registeredSplit
protected SourceConfiguration	sourceConfiguration
protected AtomicBoolean	wakeup

Constructor Summary

Constructors

Constructor and Description
PulsarOrderedPartitionSplitReader(org.apache.pulsar.client.api.PulsarClient pulsarClient, org.apache.pulsar.client.admin.PulsarAdmin pulsarAdmin, SourceConfiguration sourceConfiguration, PulsarDeserializationSchema<OUT> deserializationSchema)

Method Summary

Modifier and Type	Method and Description
void	close()
protected org.apache.pulsar.client.api.Consumer<?>	createPulsarConsumer(PulsarPartitionSplit split) Create a specified Consumer by the given split information.
protected org.apache.pulsar.client.api.Consumer<?>	createPulsarConsumer(TopicPartition partition)
org.apache.flink.connector.base.source.reader.RecordsWithSplitIds<PulsarMessage<OUT>>	fetch()
protected void	finishedPollMessage(org.apache.pulsar.client.api.Message<?> message)
void	handleSplitsChanges(org.apache.flink.connector.base.source.reader.splitreader.SplitsChange<PulsarPartitionSplit> splitsChanges)
protected boolean	isNotWakeup()
void	notifyCheckpointComplete(TopicPartition partition, org.apache.pulsar.client.api.MessageId offsetsToCommit)
protected org.apache.pulsar.client.api.Message<?>	pollMessage(java.time.Duration timeout)
protected void	startConsumer(PulsarPartitionSplit split, org.apache.pulsar.client.api.Consumer<?> consumer)
void	wakeUp()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

pulsarClient

protected final org.apache.pulsar.client.api.PulsarClient pulsarClient

pulsarAdmin

protected final org.apache.pulsar.client.admin.PulsarAdmin pulsarAdmin

sourceConfiguration

protected final SourceConfiguration sourceConfiguration

deserializationSchema

protected final PulsarDeserializationSchema<OUT> deserializationSchema

wakeup

protected final AtomicBoolean wakeup

pulsarConsumer

protected org.apache.pulsar.client.api.Consumer<?> pulsarConsumer

registeredSplit

protected PulsarPartitionSplit registeredSplit

Constructor Detail

PulsarOrderedPartitionSplitReader

public PulsarOrderedPartitionSplitReader(org.apache.pulsar.client.api.PulsarClient pulsarClient,
                                         org.apache.pulsar.client.admin.PulsarAdmin pulsarAdmin,
                                         SourceConfiguration sourceConfiguration,
                                         PulsarDeserializationSchema<OUT> deserializationSchema)

Method Detail

pollMessage

protected org.apache.pulsar.client.api.Message<?> pollMessage(java.time.Duration timeout)
                                                       throws org.apache.pulsar.client.api.PulsarClientException

Throws:

org.apache.pulsar.client.api.PulsarClientException

finishedPollMessage

protected void finishedPollMessage(org.apache.pulsar.client.api.Message<?> message)

startConsumer

protected void startConsumer(PulsarPartitionSplit split,
                             org.apache.pulsar.client.api.Consumer<?> consumer)

notifyCheckpointComplete

public void notifyCheckpointComplete(TopicPartition partition,
                                     org.apache.pulsar.client.api.MessageId offsetsToCommit)

fetch

public org.apache.flink.connector.base.source.reader.RecordsWithSplitIds<PulsarMessage<OUT>> fetch()
                                                                                            throws IOException

Specified by:

fetch in interface org.apache.flink.connector.base.source.reader.splitreader.SplitReader<PulsarMessage<OUT>,PulsarPartitionSplit>

Throws:

IOException

handleSplitsChanges

public void handleSplitsChanges(org.apache.flink.connector.base.source.reader.splitreader.SplitsChange<PulsarPartitionSplit> splitsChanges)

Specified by:

handleSplitsChanges in interface org.apache.flink.connector.base.source.reader.splitreader.SplitReader<PulsarMessage<OUT>,PulsarPartitionSplit>

wakeUp

public void wakeUp()

Specified by:

wakeUp in interface org.apache.flink.connector.base.source.reader.splitreader.SplitReader<PulsarMessage<OUT>,PulsarPartitionSplit>

close

public void close()

Specified by:

close in interface org.apache.flink.connector.base.source.reader.splitreader.SplitReader<PulsarMessage<OUT>,PulsarPartitionSplit>

isNotWakeup

protected boolean isNotWakeup()

createPulsarConsumer

protected org.apache.pulsar.client.api.Consumer<?> createPulsarConsumer(PulsarPartitionSplit split)

Create a specified Consumer by the given split information. If using pulsar schema, then use the pulsar schema, if using flink schema, then use a Schema.BYTES

createPulsarConsumer

protected org.apache.pulsar.client.api.Consumer<?> createPulsarConsumer(TopicPartition partition)