org.apache.flink.streaming.connectors.pulsar.table

Class PulsarDynamicTableSource

java.lang.Object

org.apache.flink.streaming.connectors.pulsar.table.PulsarDynamicTableSource

All Implemented Interfaces:

org.apache.flink.table.connector.source.abilities.SupportsReadingMetadata, org.apache.flink.table.connector.source.abilities.SupportsWatermarkPushDown, org.apache.flink.table.connector.source.DynamicTableSource, org.apache.flink.table.connector.source.ScanTableSource

public class PulsarDynamicTableSource
extends Object
implements org.apache.flink.table.connector.source.ScanTableSource, org.apache.flink.table.connector.source.abilities.SupportsReadingMetadata, org.apache.flink.table.connector.source.abilities.SupportsWatermarkPushDown

pulsar dynamic table source.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.flink.table.connector.source.ScanTableSource

org.apache.flink.table.connector.source.ScanTableSource.ScanContext, org.apache.flink.table.connector.source.ScanTableSource.ScanRuntimeProvider

Nested classes/interfaces inherited from interface org.apache.flink.table.connector.source.DynamicTableSource

org.apache.flink.table.connector.source.DynamicTableSource.Context, org.apache.flink.table.connector.source.DynamicTableSource.DataStructureConverter

Field Summary

Fields

Modifier and Type	Field and Description
protected String	adminUrl The Pulsar topic to consume.
protected org.apache.flink.table.connector.format.DecodingFormat<org.apache.flink.api.common.serialization.DeserializationSchema<org.apache.flink.table.data.RowData>>	keyDecodingFormat Optional format for decoding keys from Pulsar.
protected String	keyPrefix Prefix that needs to be removed from fields when constructing the physical data type.
protected int[]	keyProjection Indices that determine the key fields and the target position in the produced row.
protected List<String>	metadataKeys Metadata that is appended at the end of a physical source row.
protected org.apache.flink.table.types.DataType	physicalDataType Data type to configure the formats.
protected org.apache.flink.table.types.DataType	producedDataType Data type that describes the final output of the source.
protected Properties	properties Properties for the Pulsar consumer.
protected String	serviceUrl The Pulsar topic to consume.
protected PulsarTableOptions.StartupOptions	startupOptions The startup mode for the contained consumer (default is StartupMode.LATEST).
protected String	topicPattern The Pulsar topic to consume.
protected List<String>	topics The Pulsar topic to consume.
protected boolean	upsertMode Flag to determine source mode.
protected org.apache.flink.table.connector.format.DecodingFormat<org.apache.flink.api.common.serialization.DeserializationSchema<org.apache.flink.table.data.RowData>>	valueDecodingFormat Format for decoding values from Pulsar.
protected int[]	valueProjection Indices that determine the value fields and the target position in the produced row.
protected org.apache.flink.api.common.eventtime.WatermarkStrategy<org.apache.flink.table.data.RowData>	watermarkStrategy Watermark strategy that is used to generate per-partition watermark.

Constructor Summary

Constructors

Constructor and Description

PulsarDynamicTableSource(org.apache.flink.table.types.DataType physicalDataType, org.apache.flink.table.connector.format.DecodingFormat<org.apache.flink.api.common.serialization.DeserializationSchema<org.apache.flink.table.data.RowData>> keyDecodingFormat, org.apache.flink.table.connector.format.DecodingFormat<org.apache.flink.api.common.serialization.DeserializationSchema<org.apache.flink.table.data.RowData>> valueDecodingFormat, int[] keyProjection, int[] valueProjection, String keyPrefix, List<String> topics, String topicPattern, String serviceUrl, String adminUrl, Properties properties, PulsarTableOptions.StartupOptions startupOptions, boolean upsertMode)

Method Summary

Modifier and Type	Method and Description
void	applyReadableMetadata(List<String> metadataKeys, org.apache.flink.table.types.DataType producedDataType)
void	applyWatermark(org.apache.flink.api.common.eventtime.WatermarkStrategy<org.apache.flink.table.data.RowData> watermarkStrategy)
String	asSummaryString()
org.apache.flink.table.connector.source.DynamicTableSource	copy()
boolean	equals(Object o)
org.apache.flink.table.connector.ChangelogMode	getChangelogMode()
org.apache.flink.table.connector.source.ScanTableSource.ScanRuntimeProvider	getScanRuntimeProvider(org.apache.flink.table.connector.source.ScanTableSource.ScanContext context)
int	hashCode()
Map<String,org.apache.flink.table.types.DataType>	listReadableMetadata()

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, toString, wait, wait, wait

Field Detail

producedDataType

protected org.apache.flink.table.types.DataType producedDataType

Data type that describes the final output of the source.

metadataKeys

protected List<String> metadataKeys

Metadata that is appended at the end of a physical source row.

watermarkStrategy

@Nullable
protected org.apache.flink.api.common.eventtime.WatermarkStrategy<org.apache.flink.table.data.RowData> watermarkStrategy

Watermark strategy that is used to generate per-partition watermark.

physicalDataType

protected final org.apache.flink.table.types.DataType physicalDataType

Data type to configure the formats.

keyDecodingFormat

@Nullable
protected final org.apache.flink.table.connector.format.DecodingFormat<org.apache.flink.api.common.serialization.DeserializationSchema<org.apache.flink.table.data.RowData>> keyDecodingFormat

Optional format for decoding keys from Pulsar.

valueDecodingFormat

protected final org.apache.flink.table.connector.format.DecodingFormat<org.apache.flink.api.common.serialization.DeserializationSchema<org.apache.flink.table.data.RowData>> valueDecodingFormat

Format for decoding values from Pulsar.

keyProjection

protected final int[] keyProjection

Indices that determine the key fields and the target position in the produced row.

valueProjection

protected final int[] valueProjection

Indices that determine the value fields and the target position in the produced row.

keyPrefix

@Nullable
protected final String keyPrefix

Prefix that needs to be removed from fields when constructing the physical data type.

topics

protected final List<String> topics

The Pulsar topic to consume.

topicPattern

protected final String topicPattern

The Pulsar topic to consume.

serviceUrl

protected final String serviceUrl

The Pulsar topic to consume.

adminUrl

protected final String adminUrl

The Pulsar topic to consume.

properties

protected final Properties properties

Properties for the Pulsar consumer.

startupOptions

protected final PulsarTableOptions.StartupOptions startupOptions

The startup mode for the contained consumer (default is StartupMode.LATEST).

upsertMode

protected final boolean upsertMode

Flag to determine source mode. In upsert mode, it will keep the tombstone message.

Constructor Detail

PulsarDynamicTableSource

public PulsarDynamicTableSource(org.apache.flink.table.types.DataType physicalDataType,
                                @Nullable
                                org.apache.flink.table.connector.format.DecodingFormat<org.apache.flink.api.common.serialization.DeserializationSchema<org.apache.flink.table.data.RowData>> keyDecodingFormat,
                                org.apache.flink.table.connector.format.DecodingFormat<org.apache.flink.api.common.serialization.DeserializationSchema<org.apache.flink.table.data.RowData>> valueDecodingFormat,
                                int[] keyProjection,
                                int[] valueProjection,
                                @Nullable
                                String keyPrefix,
                                List<String> topics,
                                String topicPattern,
                                String serviceUrl,
                                String adminUrl,
                                Properties properties,
                                PulsarTableOptions.StartupOptions startupOptions,
                                boolean upsertMode)

Method Detail

getChangelogMode

public org.apache.flink.table.connector.ChangelogMode getChangelogMode()

Specified by:

getChangelogMode in interface org.apache.flink.table.connector.source.ScanTableSource

getScanRuntimeProvider

public org.apache.flink.table.connector.source.ScanTableSource.ScanRuntimeProvider getScanRuntimeProvider(org.apache.flink.table.connector.source.ScanTableSource.ScanContext context)

Specified by:

getScanRuntimeProvider in interface org.apache.flink.table.connector.source.ScanTableSource

copy

public org.apache.flink.table.connector.source.DynamicTableSource copy()

Specified by:

copy in interface org.apache.flink.table.connector.source.DynamicTableSource

asSummaryString

public String asSummaryString()

Specified by:

asSummaryString in interface org.apache.flink.table.connector.source.DynamicTableSource

equals

public boolean equals(Object o)

Overrides:

equals in class Object

hashCode

public int hashCode()

Overrides:

hashCode in class Object

listReadableMetadata

public Map<String,org.apache.flink.table.types.DataType> listReadableMetadata()

Specified by:

listReadableMetadata in interface org.apache.flink.table.connector.source.abilities.SupportsReadingMetadata

applyReadableMetadata

public void applyReadableMetadata(List<String> metadataKeys,
                                  org.apache.flink.table.types.DataType producedDataType)

Specified by:

applyReadableMetadata in interface org.apache.flink.table.connector.source.abilities.SupportsReadingMetadata

applyWatermark

public void applyWatermark(org.apache.flink.api.common.eventtime.WatermarkStrategy<org.apache.flink.table.data.RowData> watermarkStrategy)

Specified by:

applyWatermark in interface org.apache.flink.table.connector.source.abilities.SupportsWatermarkPushDown