org.apache.flink.streaming.connectors.pulsar.table

Class PulsarDynamicTableSink

java.lang.Object

org.apache.flink.streaming.connectors.pulsar.table.PulsarDynamicTableSink

All Implemented Interfaces:

org.apache.flink.table.connector.sink.abilities.SupportsWritingMetadata, org.apache.flink.table.connector.sink.DynamicTableSink

public class PulsarDynamicTableSink
extends Object
implements org.apache.flink.table.connector.sink.DynamicTableSink, org.apache.flink.table.connector.sink.abilities.SupportsWritingMetadata

pulsar dynamic table sink.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.flink.table.connector.sink.DynamicTableSink

org.apache.flink.table.connector.sink.DynamicTableSink.Context, org.apache.flink.table.connector.sink.DynamicTableSink.DataStructureConverter, org.apache.flink.table.connector.sink.DynamicTableSink.SinkRuntimeProvider

Field Summary

Fields

Modifier and Type	Field and Description
protected String	adminUrl
protected org.apache.flink.table.connector.format.EncodingFormat<org.apache.flink.api.common.serialization.SerializationSchema<org.apache.flink.table.data.RowData>>	keyEncodingFormat Optional format for encoding keys to Pulsar.
protected String	keyPrefix Prefix that needs to be removed from fields when constructing the physical data type.
protected int[]	keyProjection Indices that determine the key fields and the source position in the consumed row.
protected List<String>	metadataKeys Metadata that is appended at the end of a physical sink row.
protected Integer	parallelism Parallelism of the physical Pulsar producer.
protected org.apache.flink.table.types.DataType	physicalDataType Data type to configure the formats.
protected Properties	properties Properties for the pulsar producer.
protected PulsarSinkSemantic	semantic Sink commit semantic.
protected String	serviceUrl
protected String	topic The pulsar topic to write to.
protected boolean	upsertMode Flag to determine sink mode.
protected org.apache.flink.table.connector.format.EncodingFormat<org.apache.flink.api.common.serialization.SerializationSchema<org.apache.flink.table.data.RowData>>	valueEncodingFormat Sink format for encoding records to pulsar.
protected int[]	valueProjection Indices that determine the value fields and the source position in the consumed row.

Constructor Summary

Constructors

Modifier

Constructor and Description

protected

PulsarDynamicTableSink(String serviceUrl, String adminUrl, String topic, org.apache.flink.table.types.DataType physicalDataType, Properties properties, org.apache.flink.table.connector.format.EncodingFormat<org.apache.flink.api.common.serialization.SerializationSchema<org.apache.flink.table.data.RowData>> keyEncodingFormat, org.apache.flink.table.connector.format.EncodingFormat<org.apache.flink.api.common.serialization.SerializationSchema<org.apache.flink.table.data.RowData>> valueEncodingFormat, int[] keyProjection, int[] valueProjection, String keyPrefix, PulsarSinkSemantic semantic, String formatType, boolean upsertMode, Integer parallelism, org.apache.pulsar.client.api.MessageRouter messageRouter)

Method Summary

Modifier and Type	Method and Description
void	applyWritableMetadata(List<String> metadataKeys, org.apache.flink.table.types.DataType consumedDataType)
String	asSummaryString()
org.apache.flink.table.connector.sink.DynamicTableSink	copy()
boolean	equals(Object o)
org.apache.flink.table.connector.ChangelogMode	getChangelogMode(org.apache.flink.table.connector.ChangelogMode requestedMode)
org.apache.pulsar.client.api.MessageRouter	getMessageRouter()
org.apache.flink.table.connector.sink.DynamicTableSink.SinkRuntimeProvider	getSinkRuntimeProvider(org.apache.flink.table.connector.sink.DynamicTableSink.Context context)
int	hashCode()
Map<String,org.apache.flink.table.types.DataType>	listWritableMetadata()

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, toString, wait, wait, wait

Field Detail

metadataKeys

protected List<String> metadataKeys

Metadata that is appended at the end of a physical sink row.

physicalDataType

protected final org.apache.flink.table.types.DataType physicalDataType

Data type to configure the formats.

topic

protected final String topic

The pulsar topic to write to.

serviceUrl

protected final String serviceUrl

adminUrl

protected final String adminUrl

properties

protected final Properties properties

Properties for the pulsar producer.

keyEncodingFormat

@Nullable
protected final org.apache.flink.table.connector.format.EncodingFormat<org.apache.flink.api.common.serialization.SerializationSchema<org.apache.flink.table.data.RowData>> keyEncodingFormat

Optional format for encoding keys to Pulsar.

valueEncodingFormat

protected final org.apache.flink.table.connector.format.EncodingFormat<org.apache.flink.api.common.serialization.SerializationSchema<org.apache.flink.table.data.RowData>> valueEncodingFormat

Sink format for encoding records to pulsar.

keyProjection

protected final int[] keyProjection

Indices that determine the key fields and the source position in the consumed row.

valueProjection

protected final int[] valueProjection

Indices that determine the value fields and the source position in the consumed row.

keyPrefix

@Nullable
protected final String keyPrefix

Prefix that needs to be removed from fields when constructing the physical data type.

upsertMode

protected final boolean upsertMode

Flag to determine sink mode. In upsert mode sink transforms the delete/update-before message to tombstone message.

parallelism

@Nullable
protected final Integer parallelism

Parallelism of the physical Pulsar producer.

semantic

protected final PulsarSinkSemantic semantic

Sink commit semantic.

Constructor Detail

PulsarDynamicTableSink

protected PulsarDynamicTableSink(String serviceUrl,
                                 String adminUrl,
                                 String topic,
                                 org.apache.flink.table.types.DataType physicalDataType,
                                 Properties properties,
                                 @Nullable
                                 org.apache.flink.table.connector.format.EncodingFormat<org.apache.flink.api.common.serialization.SerializationSchema<org.apache.flink.table.data.RowData>> keyEncodingFormat,
                                 org.apache.flink.table.connector.format.EncodingFormat<org.apache.flink.api.common.serialization.SerializationSchema<org.apache.flink.table.data.RowData>> valueEncodingFormat,
                                 int[] keyProjection,
                                 int[] valueProjection,
                                 @Nullable
                                 String keyPrefix,
                                 PulsarSinkSemantic semantic,
                                 String formatType,
                                 boolean upsertMode,
                                 @Nullable
                                 Integer parallelism,
                                 @Nullable
                                 org.apache.pulsar.client.api.MessageRouter messageRouter)

Method Detail

getChangelogMode

public org.apache.flink.table.connector.ChangelogMode getChangelogMode(org.apache.flink.table.connector.ChangelogMode requestedMode)

Specified by:

getChangelogMode in interface org.apache.flink.table.connector.sink.DynamicTableSink

getSinkRuntimeProvider

public org.apache.flink.table.connector.sink.DynamicTableSink.SinkRuntimeProvider getSinkRuntimeProvider(org.apache.flink.table.connector.sink.DynamicTableSink.Context context)

Specified by:

getSinkRuntimeProvider in interface org.apache.flink.table.connector.sink.DynamicTableSink

getMessageRouter

public org.apache.pulsar.client.api.MessageRouter getMessageRouter()

copy

public org.apache.flink.table.connector.sink.DynamicTableSink copy()

Specified by:

copy in interface org.apache.flink.table.connector.sink.DynamicTableSink

equals

public boolean equals(Object o)

Overrides:

equals in class Object

hashCode

public int hashCode()

Overrides:

hashCode in class Object

asSummaryString

public String asSummaryString()

Specified by:

asSummaryString in interface org.apache.flink.table.connector.sink.DynamicTableSink

listWritableMetadata

public Map<String,org.apache.flink.table.types.DataType> listWritableMetadata()

Specified by:

listWritableMetadata in interface org.apache.flink.table.connector.sink.abilities.SupportsWritingMetadata

applyWritableMetadata

public void applyWritableMetadata(List<String> metadataKeys,
                                  org.apache.flink.table.types.DataType consumedDataType)

Specified by:

applyWritableMetadata in interface org.apache.flink.table.connector.sink.abilities.SupportsWritingMetadata