org.apache.flink.streaming.connectors.pulsar

Class PulsarTableSink

java.lang.Object

org.apache.flink.streaming.connectors.pulsar.PulsarTableSink

All Implemented Interfaces:

SupportsWritingMetadata, org.apache.flink.table.sinks.AppendStreamTableSink<org.apache.flink.types.Row>, org.apache.flink.table.sinks.StreamTableSink<org.apache.flink.types.Row>, org.apache.flink.table.sinks.TableSink<org.apache.flink.types.Row>

public class PulsarTableSink
extends Object
implements org.apache.flink.table.sinks.AppendStreamTableSink<org.apache.flink.types.Row>, SupportsWritingMetadata

Pulsar Table Sink.

Field Summary

Fields

Modifier and Type	Field and Description
protected List<String>	metadataKeys Metadata that is appended at the end of a physical sink row.
protected org.apache.flink.table.types.DataType	physicalDataType Data type to configure the format.
protected boolean	useExtendField

Constructor Summary

Constructors

Constructor and Description
PulsarTableSink(String serviceUrl, String adminUrl, org.apache.flink.table.api.TableSchema schema, String defaultTopicName, Properties properties, org.apache.flink.api.common.serialization.SerializationSchema serializationSchema)
PulsarTableSink(String adminUrl, org.apache.flink.table.api.TableSchema schema, String defaultTopicName, org.apache.pulsar.client.impl.conf.ClientConfigurationData clientConf, Properties properties, org.apache.flink.api.common.serialization.SerializationSchema serializationSchema)

Method Summary

Modifier and Type	Method and Description
void	applyWritableMetadata(List<String> metadataKeys, org.apache.flink.table.types.DataType consumedDataType) Provides a list of metadata keys that the consumed RowData will contain as appended metadata columns which must be persisted.
org.apache.flink.table.sinks.TableSink<org.apache.flink.types.Row>	configure(String[] fieldNames, org.apache.flink.api.common.typeinfo.TypeInformation<?>[] fieldTypes)
org.apache.flink.streaming.api.datastream.DataStreamSink<?>	consumeDataStream(org.apache.flink.streaming.api.datastream.DataStream<org.apache.flink.types.Row> dataStream)
void	emitDataStream(org.apache.flink.streaming.api.datastream.DataStream<org.apache.flink.types.Row> dataStream)
String[]	getFieldNames()
org.apache.flink.api.common.typeinfo.TypeInformation<?>[]	getFieldTypes()
org.apache.flink.api.common.typeinfo.TypeInformation<org.apache.flink.types.Row>	getOutputType()
org.apache.flink.table.api.TableSchema	getTableSchema()
Map<String,org.apache.flink.table.types.DataType>	listWritableMetadata() Returns the map of metadata keys and their corresponding data types that can be consumed by this table sink for writing.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.table.sinks.TableSink

getConsumedDataType

Field Detail

metadataKeys

protected List<String> metadataKeys

Metadata that is appended at the end of a physical sink row.

physicalDataType

protected org.apache.flink.table.types.DataType physicalDataType

Data type to configure the format.

useExtendField

protected final boolean useExtendField

Constructor Detail

PulsarTableSink

public PulsarTableSink(String adminUrl,
                       org.apache.flink.table.api.TableSchema schema,
                       String defaultTopicName,
                       org.apache.pulsar.client.impl.conf.ClientConfigurationData clientConf,
                       Properties properties,
                       org.apache.flink.api.common.serialization.SerializationSchema serializationSchema)

PulsarTableSink

public PulsarTableSink(String serviceUrl,
                       String adminUrl,
                       org.apache.flink.table.api.TableSchema schema,
                       String defaultTopicName,
                       Properties properties,
                       org.apache.flink.api.common.serialization.SerializationSchema serializationSchema)

Method Detail

emitDataStream

public void emitDataStream(org.apache.flink.streaming.api.datastream.DataStream<org.apache.flink.types.Row> dataStream)

consumeDataStream

public org.apache.flink.streaming.api.datastream.DataStreamSink<?> consumeDataStream(org.apache.flink.streaming.api.datastream.DataStream<org.apache.flink.types.Row> dataStream)

Specified by:

consumeDataStream in interface org.apache.flink.table.sinks.StreamTableSink<org.apache.flink.types.Row>

getOutputType

public org.apache.flink.api.common.typeinfo.TypeInformation<org.apache.flink.types.Row> getOutputType()

Specified by:

getOutputType in interface org.apache.flink.table.sinks.TableSink<org.apache.flink.types.Row>

getTableSchema

public org.apache.flink.table.api.TableSchema getTableSchema()

Specified by:

getTableSchema in interface org.apache.flink.table.sinks.TableSink<org.apache.flink.types.Row>

getFieldNames

public String[] getFieldNames()

Specified by:

getFieldNames in interface org.apache.flink.table.sinks.TableSink<org.apache.flink.types.Row>

getFieldTypes

public org.apache.flink.api.common.typeinfo.TypeInformation<?>[] getFieldTypes()

Specified by:

getFieldTypes in interface org.apache.flink.table.sinks.TableSink<org.apache.flink.types.Row>

configure

public org.apache.flink.table.sinks.TableSink<org.apache.flink.types.Row> configure(String[] fieldNames,
                                                                                    org.apache.flink.api.common.typeinfo.TypeInformation<?>[] fieldTypes)

Specified by:

configure in interface org.apache.flink.table.sinks.TableSink<org.apache.flink.types.Row>

listWritableMetadata

public Map<String,org.apache.flink.table.types.DataType> listWritableMetadata()

Description copied from interface: SupportsWritingMetadata

Returns the map of metadata keys and their corresponding data types that can be consumed by this table sink for writing.

The returned map will be used by the planner for validation and insertion of explicit casts (see LogicalTypeCasts.supportsExplicitCast(LogicalType, LogicalType)) if necessary.

The iteration order of the returned map determines the order of metadata keys in the list passed in SupportsWritingMetadata.applyWritableMetadata(List, DataType). Therefore, it might be beneficial to return a LinkedHashMap if a strict metadata column order is required.

If a sink forwards metadata to one or more formats, we recommend the following column order for consistency:

   KEY FORMAT METADATA COLUMNS + VALUE FORMAT METADATA COLUMNS + SINK METADATA COLUMNS
 

Metadata key names follow the same pattern as mentioned in Factory. In case of duplicate names in format and sink keys, format keys shall have higher precedence.

Regardless of the returned DataTypes, a metadata column is always represented using internal data structures (see RowData).

Specified by:

listWritableMetadata in interface SupportsWritingMetadata

applyWritableMetadata

public void applyWritableMetadata(List<String> metadataKeys,
                                  org.apache.flink.table.types.DataType consumedDataType)

Description copied from interface: SupportsWritingMetadata

Provides a list of metadata keys that the consumed RowData will contain as appended metadata columns which must be persisted.

Specified by:

applyWritableMetadata in interface SupportsWritingMetadata

Parameters:

metadataKeys - a subset of the keys returned by SupportsWritingMetadata.listWritableMetadata(), ordered by the iteration order of returned map

consumedDataType - the final input type of the sink