org.apache.flink.streaming.connectors.pulsar

Interface SupportsReadingMetadata

All Known Implementing Classes:

PulsarDynamicTableSource, PulsarTableSource

@PublicEvolving
public interface SupportsReadingMetadata

Before Flink 1.12, we simulated this interface, after which we will use the official interface.

Method Summary

Modifier and Type	Method and Description
void	applyReadableMetadata(List<String> metadataKeys, org.apache.flink.table.types.DataType producedDataType) Provides a list of metadata keys that the produced RowData must contain as appended metadata columns.
Map<String,org.apache.flink.table.types.DataType>	listReadableMetadata() Returns the map of metadata keys and their corresponding data types that can be produced by this table source for reading.

Method Detail

listReadableMetadata

Map<String,org.apache.flink.table.types.DataType> listReadableMetadata()

Returns the map of metadata keys and their corresponding data types that can be produced by this table source for reading.

The returned map will be used by the planner for validation and insertion of explicit casts (see LogicalTypeCasts.supportsExplicitCast(LogicalType, LogicalType)) if necessary.

The iteration order of the returned map determines the order of metadata keys in the list passed in applyReadableMetadata(List, DataType). Therefore, it might be beneficial to return a LinkedHashMap if a strict metadata column order is required.

If a source forwards metadata from one or more formats, we recommend the following column order for consistency:

   KEY FORMAT METADATA COLUMNS + VALUE FORMAT METADATA COLUMNS + SOURCE METADATA COLUMNS
 

Metadata key names follow the same pattern as mentioned in Factory. In case of duplicate names in format and source keys, format keys shall have higher precedence.

Regardless of the returned DataTypes, a metadata column is always represented using internal data structures (see RowData).

applyReadableMetadata

void applyReadableMetadata(List<String> metadataKeys,
                           org.apache.flink.table.types.DataType producedDataType)

Provides a list of metadata keys that the produced RowData must contain as appended metadata columns.

Note: Use the passed data type instead of TableSchema.toPhysicalRowDataType() for describing the final output data type when creating TypeInformation. If the source implements SupportsProjectionPushDown, the projection is already considered in the given output data type.

Parameters:

metadataKeys - a subset of the keys returned by listReadableMetadata(), ordered by the iteration order of returned map

producedDataType - the final output type of the source