org.apache.hudi.table

Class HoodieTableSource

java.lang.Object

org.apache.hudi.table.HoodieTableSource

All Implemented Interfaces:

org.apache.flink.table.connector.source.abilities.SupportsFilterPushDown, org.apache.flink.table.connector.source.abilities.SupportsLimitPushDown, org.apache.flink.table.connector.source.abilities.SupportsProjectionPushDown, org.apache.flink.table.connector.source.DynamicTableSource, org.apache.flink.table.connector.source.ScanTableSource

public class HoodieTableSource
extends Object
implements org.apache.flink.table.connector.source.ScanTableSource, org.apache.flink.table.connector.source.abilities.SupportsProjectionPushDown, org.apache.flink.table.connector.source.abilities.SupportsLimitPushDown, org.apache.flink.table.connector.source.abilities.SupportsFilterPushDown

Hoodie batch table source that always read the latest snapshot of the underneath table.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.flink.table.connector.source.ScanTableSource

org.apache.flink.table.connector.source.ScanTableSource.ScanContext, org.apache.flink.table.connector.source.ScanTableSource.ScanRuntimeProvider

Nested classes/interfaces inherited from interface org.apache.flink.table.connector.source.DynamicTableSource

org.apache.flink.table.connector.source.DynamicTableSource.Context, org.apache.flink.table.connector.source.DynamicTableSource.DataStructureConverter

Nested classes/interfaces inherited from interface org.apache.flink.table.connector.source.abilities.SupportsFilterPushDown

org.apache.flink.table.connector.source.abilities.SupportsFilterPushDown.Result

Constructor Summary

Constructors

Constructor and Description
HoodieTableSource(org.apache.flink.table.catalog.ResolvedSchema schema, org.apache.hadoop.fs.Path path, List<String> partitionKeys, String defaultPartName, org.apache.flink.configuration.Configuration conf)
HoodieTableSource(org.apache.flink.table.catalog.ResolvedSchema schema, org.apache.hadoop.fs.Path path, List<String> partitionKeys, String defaultPartName, org.apache.flink.configuration.Configuration conf, List<ExpressionPredicates.Predicate> predicates, DataPruner dataPruner, PartitionPruners.PartitionPruner partitionPruner, int dataBucket, int[] requiredPos, Long limit, HoodieTableMetaClient metaClient, InternalSchemaManager internalSchemaManager)

Method Summary

Modifier and Type	Method and Description
org.apache.flink.table.connector.source.abilities.SupportsFilterPushDown.Result	applyFilters(List<org.apache.flink.table.expressions.ResolvedExpression> filters)
void	applyLimit(long limit)
void	applyProjection(int[][] projections)
String	asSummaryString()
org.apache.flink.table.connector.source.DynamicTableSource	copy()
org.apache.flink.table.connector.ChangelogMode	getChangelogMode()
org.apache.flink.configuration.Configuration	getConf()
int	getDataBucket()
DataPruner	getDataPruner()
org.apache.flink.api.common.io.InputFormat<org.apache.flink.table.data.RowData,?>	getInputFormat()
org.apache.flink.api.common.io.InputFormat<org.apache.flink.table.data.RowData,?>	getInputFormat(boolean isStreaming)
HoodieTableMetaClient	getMetaClient()
List<ExpressionPredicates.Predicate>	getPredicates()
org.apache.hadoop.fs.FileStatus[]	getReadFiles() Get the reader paths with partition path expanded.
org.apache.flink.table.connector.source.ScanTableSource.ScanRuntimeProvider	getScanRuntimeProvider(org.apache.flink.table.connector.source.ScanTableSource.ScanContext scanContext)
org.apache.avro.Schema	getTableAvroSchema()
void	reset() Reset the state of the table source.
boolean	supportsNestedProjection()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.table.connector.source.abilities.SupportsProjectionPushDown

applyProjection

Constructor Detail

HoodieTableSource

public HoodieTableSource(org.apache.flink.table.catalog.ResolvedSchema schema,
                         org.apache.hadoop.fs.Path path,
                         List<String> partitionKeys,
                         String defaultPartName,
                         org.apache.flink.configuration.Configuration conf)

HoodieTableSource

public HoodieTableSource(org.apache.flink.table.catalog.ResolvedSchema schema,
                         org.apache.hadoop.fs.Path path,
                         List<String> partitionKeys,
                         String defaultPartName,
                         org.apache.flink.configuration.Configuration conf,
                         @Nullable
                         List<ExpressionPredicates.Predicate> predicates,
                         @Nullable
                         DataPruner dataPruner,
                         @Nullable
                         PartitionPruners.PartitionPruner partitionPruner,
                         int dataBucket,
                         @Nullable
                         int[] requiredPos,
                         @Nullable
                         Long limit,
                         @Nullable
                         HoodieTableMetaClient metaClient,
                         @Nullable
                         InternalSchemaManager internalSchemaManager)

Method Detail

getScanRuntimeProvider

public org.apache.flink.table.connector.source.ScanTableSource.ScanRuntimeProvider getScanRuntimeProvider(org.apache.flink.table.connector.source.ScanTableSource.ScanContext scanContext)

Specified by:

getScanRuntimeProvider in interface org.apache.flink.table.connector.source.ScanTableSource

getChangelogMode

public org.apache.flink.table.connector.ChangelogMode getChangelogMode()

Specified by:

getChangelogMode in interface org.apache.flink.table.connector.source.ScanTableSource

copy

public org.apache.flink.table.connector.source.DynamicTableSource copy()

Specified by:

copy in interface org.apache.flink.table.connector.source.DynamicTableSource

asSummaryString

public String asSummaryString()

Specified by:

asSummaryString in interface org.apache.flink.table.connector.source.DynamicTableSource

applyFilters

public org.apache.flink.table.connector.source.abilities.SupportsFilterPushDown.Result applyFilters(List<org.apache.flink.table.expressions.ResolvedExpression> filters)

Specified by:

applyFilters in interface org.apache.flink.table.connector.source.abilities.SupportsFilterPushDown

supportsNestedProjection

public boolean supportsNestedProjection()

Specified by:

supportsNestedProjection in interface org.apache.flink.table.connector.source.abilities.SupportsProjectionPushDown

applyProjection

public void applyProjection(int[][] projections)

Specified by:

applyProjection in interface org.apache.flink.table.connector.source.abilities.SupportsProjectionPushDown

applyLimit

public void applyLimit(long limit)

Specified by:

applyLimit in interface org.apache.flink.table.connector.source.abilities.SupportsLimitPushDown

getInputFormat

public org.apache.flink.api.common.io.InputFormat<org.apache.flink.table.data.RowData,?> getInputFormat()

getInputFormat

@VisibleForTesting
public org.apache.flink.api.common.io.InputFormat<org.apache.flink.table.data.RowData,?> getInputFormat(boolean isStreaming)

getTableAvroSchema

@VisibleForTesting
public org.apache.avro.Schema getTableAvroSchema()

getMetaClient

@VisibleForTesting
public HoodieTableMetaClient getMetaClient()

getConf

@VisibleForTesting
public org.apache.flink.configuration.Configuration getConf()

reset

@VisibleForTesting
public void reset()

Reset the state of the table source.

getReadFiles

@VisibleForTesting
public org.apache.hadoop.fs.FileStatus[] getReadFiles()

Get the reader paths with partition path expanded.

getPredicates

@VisibleForTesting
public List<ExpressionPredicates.Predicate> getPredicates()

getDataPruner

@VisibleForTesting
public DataPruner getDataPruner()

getDataBucket

@VisibleForTesting
public int getDataBucket()