org.apache.flink.connector.pulsar.table.source

Class PulsarTableDeserializationSchemaFactory

java.lang.Object

org.apache.flink.connector.pulsar.table.source.PulsarTableDeserializationSchemaFactory

All Implemented Interfaces:

Serializable

public class PulsarTableDeserializationSchemaFactory
extends Object
implements Serializable

Contains key, value projection and format information, and use such information to create a PulsarTableDeserializationSchema instance used by runtime PulsarSource instance.

A Flink row fields has a strict order: Physical Fields (Key + value) + Format Metadata Fields Connector Metadata Fields. Physical Fields are fields come directly from Pulsar message body; Format Metadata Fields are from the extra information from the decoding format. Connector metadata fields are the ones most Pulsar messages have, such as publish time, message size and producer name.

In general, Physical fields + Format Metadata fields are contained in the RowData decoded using valueDecodingFormat. Only Connector Metadata fields needs to be appended to the decoded RowData. The tricky part is to put format metadata and connector metadata in the right location. This requires an explicit adjustment process.

For example, suppose Physical Fields (Key + value) + Format Metadata Fields + Connector Metadata Fields. has arity of 11, key projection is [0, 6], and physical value projection is [1, 2, 3, 4, 5], Then after the adjustment, key projection should be [0, 6], physical value projection should be [1, 2, 3, 4, 5] and format metadata projection should be [7], connector metadata projection should be [8, 9, 10].

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description

PulsarTableDeserializationSchemaFactory(org.apache.flink.table.types.DataType physicalDataType, org.apache.flink.table.connector.format.DecodingFormat<org.apache.flink.api.common.serialization.DeserializationSchema<org.apache.flink.table.data.RowData>> keyDecodingFormat, int[] keyProjection, org.apache.flink.table.connector.format.DecodingFormat<org.apache.flink.api.common.serialization.DeserializationSchema<org.apache.flink.table.data.RowData>> valueDecodingFormat, int[] valueProjection)

Method Summary

Modifier and Type	Method and Description
PulsarDeserializationSchema<org.apache.flink.table.data.RowData>	createPulsarDeserialization(org.apache.flink.table.connector.source.ScanTableSource.ScanContext context)
boolean	equals(Object o)
int	hashCode()
void	setConnectorMetadataKeys(List<String> metadataKeys)
void	setProducedDataType(org.apache.flink.table.types.DataType producedDataType)

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

PulsarTableDeserializationSchemaFactory

public PulsarTableDeserializationSchemaFactory(org.apache.flink.table.types.DataType physicalDataType,
                                               @Nullable
                                               org.apache.flink.table.connector.format.DecodingFormat<org.apache.flink.api.common.serialization.DeserializationSchema<org.apache.flink.table.data.RowData>> keyDecodingFormat,
                                               int[] keyProjection,
                                               org.apache.flink.table.connector.format.DecodingFormat<org.apache.flink.api.common.serialization.DeserializationSchema<org.apache.flink.table.data.RowData>> valueDecodingFormat,
                                               int[] valueProjection)

Method Detail

createPulsarDeserialization

public PulsarDeserializationSchema<org.apache.flink.table.data.RowData> createPulsarDeserialization(org.apache.flink.table.connector.source.ScanTableSource.ScanContext context)

setProducedDataType

public void setProducedDataType(org.apache.flink.table.types.DataType producedDataType)

setConnectorMetadataKeys

public void setConnectorMetadataKeys(List<String> metadataKeys)

equals

public boolean equals(Object o)

Overrides:

equals in class Object

hashCode

public int hashCode()

Overrides:

hashCode in class Object