org.apache.flink.streaming.connectors.pulsar

Interface SupportsWritingMetadata

All Known Implementing Classes:

PulsarDynamicTableSink, PulsarTableSink

@PublicEvolving
public interface SupportsWritingMetadata

Before Flink 1.12, we simulated this interface, after which we will use the official interface.

Method Summary

Modifier and Type	Method and Description
void	applyWritableMetadata(List<String> metadataKeys, org.apache.flink.table.types.DataType consumedDataType) Provides a list of metadata keys that the consumed RowData will contain as appended metadata columns which must be persisted.
Map<String,org.apache.flink.table.types.DataType>	listWritableMetadata() Returns the map of metadata keys and their corresponding data types that can be consumed by this table sink for writing.

Method Detail

listWritableMetadata

Map<String,org.apache.flink.table.types.DataType> listWritableMetadata()

Returns the map of metadata keys and their corresponding data types that can be consumed by this table sink for writing.

The returned map will be used by the planner for validation and insertion of explicit casts (see LogicalTypeCasts.supportsExplicitCast(LogicalType, LogicalType)) if necessary.

The iteration order of the returned map determines the order of metadata keys in the list passed in applyWritableMetadata(List, DataType). Therefore, it might be beneficial to return a LinkedHashMap if a strict metadata column order is required.

If a sink forwards metadata to one or more formats, we recommend the following column order for consistency:

   KEY FORMAT METADATA COLUMNS + VALUE FORMAT METADATA COLUMNS + SINK METADATA COLUMNS
 

Metadata key names follow the same pattern as mentioned in Factory. In case of duplicate names in format and sink keys, format keys shall have higher precedence.

Regardless of the returned DataTypes, a metadata column is always represented using internal data structures (see RowData).

applyWritableMetadata

void applyWritableMetadata(List<String> metadataKeys,
                           org.apache.flink.table.types.DataType consumedDataType)

Provides a list of metadata keys that the consumed RowData will contain as appended metadata columns which must be persisted.

Parameters:

metadataKeys - a subset of the keys returned by listWritableMetadata(), ordered by the iteration order of returned map

consumedDataType - the final input type of the sink