org.apache.flink.streaming.connectors.pulsar.internal

Class PulsarMetadataReader

java.lang.Object

org.apache.flink.streaming.connectors.pulsar.internal.PulsarMetadataReader

All Implemented Interfaces:

AutoCloseable

public class PulsarMetadataReader
extends Object
implements AutoCloseable

A Helper class that talks to Pulsar Admin API. - getEarliest / Latest / Specific MessageIds - guarantee message existence using subscription by setup, move and remove

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	PulsarMetadataReader.ClosedException Designate the close of the metadata reader.

Constructor Summary

Constructors

Constructor and Description
PulsarMetadataReader(String adminUrl, org.apache.pulsar.client.impl.conf.ClientConfigurationData clientConf, String subscriptionName, Map<String,String> caseInsensitiveParams, int indexOfThisSubtask, int numParallelSubtasks)
PulsarMetadataReader(String adminUrl, org.apache.pulsar.client.impl.conf.ClientConfigurationData clientConf, String subscriptionName, Map<String,String> caseInsensitiveParams, int indexOfThisSubtask, int numParallelSubtasks, boolean useExternalSubscription)

Method Summary

Modifier and Type	Method and Description
boolean	checkCursorAvailable(String topic, org.apache.pulsar.client.impl.MessageIdImpl startMessageId)
void	close()
void	commitOffsetToCursor(Map<TopicRange,org.apache.pulsar.client.api.MessageId> offset)
void	createNamespace(String ns)
void	createTopic(String topicName, int defaultPartitionNum)
void	deleteNamespace(String ns)
void	deleteTopic(String topicName)
Set<TopicRange>	discoverTopicChanges()
org.apache.pulsar.client.api.MessageId	getLastMessageId(String topic)
org.apache.pulsar.client.api.MessageId	getPositionFromSubscription(TopicRange topic, org.apache.pulsar.client.api.MessageId defaultPosition)
org.apache.pulsar.common.schema.SchemaInfo	getPulsarSchema(List<String> topics)
org.apache.pulsar.common.schema.SchemaInfo	getPulsarSchema(String topic)
Set<TopicRange>	getTopicPartitions()
Set<TopicRange>	getTopicPartitionsAll() Get topic partitions all, If the topic does not exist, it is created automatically ont partition to topic.
List<TopicRange>	getTopics()
List<String>	getTopics(String ns)
List<String>	listNamespaces()
boolean	namespaceExists(String ns)
void	putSchema(String topicName, org.apache.pulsar.common.schema.SchemaInfo schemaInfo)
void	removeCursor(Set<TopicRange> topics)
void	resetCursor(TopicRange topicRange, org.apache.pulsar.client.api.MessageId messageId)
void	setupCursor(Map<TopicRange,org.apache.pulsar.client.api.MessageId> offset)
void	setupCursor(Map<TopicRange,org.apache.pulsar.client.api.MessageId> offset, boolean failOnDataLoss)
boolean	topicExists(String topicName)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

PulsarMetadataReader

public PulsarMetadataReader(String adminUrl,
                            org.apache.pulsar.client.impl.conf.ClientConfigurationData clientConf,
                            String subscriptionName,
                            Map<String,String> caseInsensitiveParams,
                            int indexOfThisSubtask,
                            int numParallelSubtasks,
                            boolean useExternalSubscription)
                     throws org.apache.pulsar.client.api.PulsarClientException

Throws:

org.apache.pulsar.client.api.PulsarClientException

PulsarMetadataReader

public PulsarMetadataReader(String adminUrl,
                            org.apache.pulsar.client.impl.conf.ClientConfigurationData clientConf,
                            String subscriptionName,
                            Map<String,String> caseInsensitiveParams,
                            int indexOfThisSubtask,
                            int numParallelSubtasks)
                     throws org.apache.pulsar.client.api.PulsarClientException

Throws:

org.apache.pulsar.client.api.PulsarClientException

Method Detail

close

public void close()

Specified by:

close in interface AutoCloseable

discoverTopicChanges

public Set<TopicRange> discoverTopicChanges()
                                     throws org.apache.pulsar.client.admin.PulsarAdminException,
                                            PulsarMetadataReader.ClosedException

Throws:

org.apache.pulsar.client.admin.PulsarAdminException

PulsarMetadataReader.ClosedException

listNamespaces

public List<String> listNamespaces()
                            throws org.apache.pulsar.client.admin.PulsarAdminException

Throws:

org.apache.pulsar.client.admin.PulsarAdminException

namespaceExists

public boolean namespaceExists(String ns)
                        throws org.apache.pulsar.client.admin.PulsarAdminException

Throws:

org.apache.pulsar.client.admin.PulsarAdminException

createNamespace

public void createNamespace(String ns)
                     throws org.apache.pulsar.client.admin.PulsarAdminException

Throws:

org.apache.pulsar.client.admin.PulsarAdminException

deleteNamespace

public void deleteNamespace(String ns)
                     throws org.apache.pulsar.client.admin.PulsarAdminException

Throws:

org.apache.pulsar.client.admin.PulsarAdminException

getTopics

public List<String> getTopics(String ns)
                       throws org.apache.pulsar.client.admin.PulsarAdminException

Throws:

org.apache.pulsar.client.admin.PulsarAdminException

topicExists

public boolean topicExists(String topicName)
                    throws org.apache.pulsar.client.admin.PulsarAdminException

Throws:

org.apache.pulsar.client.admin.PulsarAdminException

deleteTopic

public void deleteTopic(String topicName)
                 throws org.apache.pulsar.client.admin.PulsarAdminException

Throws:

org.apache.pulsar.client.admin.PulsarAdminException

createTopic

public void createTopic(String topicName,
                        int defaultPartitionNum)
                 throws org.apache.pulsar.client.admin.PulsarAdminException,
                        IncompatibleSchemaException

Throws:

org.apache.pulsar.client.admin.PulsarAdminException

IncompatibleSchemaException

putSchema

public void putSchema(String topicName,
                      org.apache.pulsar.common.schema.SchemaInfo schemaInfo)
               throws IncompatibleSchemaException

Throws:

IncompatibleSchemaException

setupCursor

public void setupCursor(Map<TopicRange,org.apache.pulsar.client.api.MessageId> offset,
                        boolean failOnDataLoss)

setupCursor

public void setupCursor(Map<TopicRange,org.apache.pulsar.client.api.MessageId> offset)

commitOffsetToCursor

public void commitOffsetToCursor(Map<TopicRange,org.apache.pulsar.client.api.MessageId> offset)

removeCursor

public void removeCursor(Set<TopicRange> topics)
                  throws PulsarMetadataReader.ClosedException

Throws:

PulsarMetadataReader.ClosedException

getPositionFromSubscription

public org.apache.pulsar.client.api.MessageId getPositionFromSubscription(TopicRange topic,
                                                                          org.apache.pulsar.client.api.MessageId defaultPosition)

getPulsarSchema

public org.apache.pulsar.common.schema.SchemaInfo getPulsarSchema(List<String> topics)
                                                           throws IncompatibleSchemaException

Throws:

IncompatibleSchemaException

getPulsarSchema

public org.apache.pulsar.common.schema.SchemaInfo getPulsarSchema(String topic)

getTopicPartitions

public Set<TopicRange> getTopicPartitions()
                                   throws org.apache.pulsar.client.admin.PulsarAdminException

Throws:

org.apache.pulsar.client.admin.PulsarAdminException

getTopicPartitionsAll

public Set<TopicRange> getTopicPartitionsAll()
                                      throws org.apache.pulsar.client.admin.PulsarAdminException

Get topic partitions all, If the topic does not exist, it is created automatically ont partition to topic.

Returns:

allTopicPartitions

Throws:

org.apache.pulsar.client.admin.PulsarAdminException - pulsarAdminException

getTopics

public List<TopicRange> getTopics()
                           throws org.apache.pulsar.client.admin.PulsarAdminException

Throws:

org.apache.pulsar.client.admin.PulsarAdminException

getLastMessageId

public org.apache.pulsar.client.api.MessageId getLastMessageId(String topic)

checkCursorAvailable

public boolean checkCursorAvailable(String topic,
                                    org.apache.pulsar.client.impl.MessageIdImpl startMessageId)

resetCursor

public void resetCursor(TopicRange topicRange,
                        org.apache.pulsar.client.api.MessageId messageId)