org.apache.flink.connector.hbase.source

Class AbstractHBaseTableSource

java.lang.Object

org.apache.flink.connector.hbase.source.AbstractHBaseTableSource

All Implemented Interfaces:

org.apache.flink.table.sources.BatchTableSource<org.apache.flink.types.Row>, org.apache.flink.table.sources.LookupableTableSource<org.apache.flink.types.Row>, org.apache.flink.table.sources.ProjectableTableSource<org.apache.flink.types.Row>, org.apache.flink.table.sources.StreamTableSource<org.apache.flink.types.Row>, org.apache.flink.table.sources.TableSource<org.apache.flink.types.Row>

@Internal
public abstract class AbstractHBaseTableSource
extends Object
implements org.apache.flink.table.sources.BatchTableSource<org.apache.flink.types.Row>, org.apache.flink.table.sources.ProjectableTableSource<org.apache.flink.types.Row>, org.apache.flink.table.sources.StreamTableSource<org.apache.flink.types.Row>, org.apache.flink.table.sources.LookupableTableSource<org.apache.flink.types.Row>

Creates a TableSource to scan an HBase table.

The table name and required HBase configuration is passed during AbstractHBaseTableSource construction. Use addColumn(String, String, Class) to specify the family, qualifier, and type of columns to scan.

The TableSource returns Row with nested Rows for each column family.

The HBaseTableSource is used as shown in the example below.

 HBaseTableSource hSrc = new HBaseTableSource(conf, "hTable");
 hSrc.setRowKey("rowkey", String.class);
 hSrc.addColumn("fam1", "col1", byte[].class);
 hSrc.addColumn("fam1", "col2", Integer.class);
 hSrc.addColumn("fam2", "col1", String.class);

 tableEnv.registerTableSourceInternal("hTable", hSrc);
 Table res = tableEnv.sqlQuery(
   "SELECT t.fam2.col1, SUM(t.fam1.col2) FROM hTable AS t " +
   "WHERE t.rowkey LIKE 'flink%' GROUP BY t.fam2.col1");
 

Field Summary

Fields

Modifier and Type	Field and Description
protected org.apache.hadoop.conf.Configuration	conf
protected HBaseTableSchema	hbaseSchema
protected String	tableName

Constructor Summary

Constructors

Constructor and Description
AbstractHBaseTableSource(org.apache.hadoop.conf.Configuration conf, String tableName, HBaseTableSchema hbaseSchema, int[] projectFields)

Method Summary

Modifier and Type	Method and Description
void	addColumn(String family, String qualifier, Class<?> clazz) Adds a column defined by family, qualifier, and type to the table schema.
String	explainSource()
org.apache.flink.table.functions.AsyncTableFunction<org.apache.flink.types.Row>	getAsyncLookupFunction(String[] lookupKeys)
org.apache.flink.api.java.DataSet<org.apache.flink.types.Row>	getDataSet(org.apache.flink.api.java.ExecutionEnvironment execEnv)
org.apache.flink.streaming.api.datastream.DataStream<org.apache.flink.types.Row>	getDataStream(org.apache.flink.streaming.api.environment.StreamExecutionEnvironment execEnv)
HBaseTableSchema	getHBaseTableSchema()
protected abstract org.apache.flink.api.common.io.InputFormat<org.apache.flink.types.Row,?>	getInputFormat(HBaseTableSchema projectedSchema)
org.apache.flink.table.functions.TableFunction<org.apache.flink.types.Row>	getLookupFunction(String[] lookupKeys)
org.apache.flink.api.common.typeinfo.TypeInformation<org.apache.flink.types.Row>	getReturnType()
org.apache.flink.table.api.TableSchema	getTableSchema()
boolean	isAsyncEnabled()
boolean	isBounded()
void	setCharset(String charset) Specifies the charset to parse Strings to HBase byte[] keys and String values.
void	setRowKey(String rowKeyName, Class<?> clazz) Sets row key information in the table schema.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.table.sources.ProjectableTableSource

projectFields

Methods inherited from interface org.apache.flink.table.sources.TableSource

getProducedDataType

Field Detail

conf

protected final org.apache.hadoop.conf.Configuration conf

tableName

protected final String tableName

hbaseSchema

protected final HBaseTableSchema hbaseSchema

Constructor Detail

AbstractHBaseTableSource

public AbstractHBaseTableSource(org.apache.hadoop.conf.Configuration conf,
                                String tableName,
                                HBaseTableSchema hbaseSchema,
                                int[] projectFields)

Method Detail

addColumn

public void addColumn(String family,
                      String qualifier,
                      Class<?> clazz)

Adds a column defined by family, qualifier, and type to the table schema.

Parameters:

family - the family name

qualifier - the qualifier name

clazz - the data type of the qualifier

setRowKey

public void setRowKey(String rowKeyName,
                      Class<?> clazz)

Sets row key information in the table schema.

Parameters:

rowKeyName - the row key field name

clazz - the data type of the row key

setCharset

public void setCharset(String charset)

Specifies the charset to parse Strings to HBase byte[] keys and String values.

Parameters:

charset - Name of the charset to use.

getReturnType

public org.apache.flink.api.common.typeinfo.TypeInformation<org.apache.flink.types.Row> getReturnType()

Specified by:

getReturnType in interface org.apache.flink.table.sources.TableSource<org.apache.flink.types.Row>

getTableSchema

public org.apache.flink.table.api.TableSchema getTableSchema()

Specified by:

getTableSchema in interface org.apache.flink.table.sources.TableSource<org.apache.flink.types.Row>

getDataSet

public org.apache.flink.api.java.DataSet<org.apache.flink.types.Row> getDataSet(org.apache.flink.api.java.ExecutionEnvironment execEnv)

Specified by:

getDataSet in interface org.apache.flink.table.sources.BatchTableSource<org.apache.flink.types.Row>

explainSource

public String explainSource()

Specified by:

explainSource in interface org.apache.flink.table.sources.TableSource<org.apache.flink.types.Row>

getLookupFunction

public org.apache.flink.table.functions.TableFunction<org.apache.flink.types.Row> getLookupFunction(String[] lookupKeys)

Specified by:

getLookupFunction in interface org.apache.flink.table.sources.LookupableTableSource<org.apache.flink.types.Row>

getAsyncLookupFunction

public org.apache.flink.table.functions.AsyncTableFunction<org.apache.flink.types.Row> getAsyncLookupFunction(String[] lookupKeys)

Specified by:

getAsyncLookupFunction in interface org.apache.flink.table.sources.LookupableTableSource<org.apache.flink.types.Row>

isAsyncEnabled

public boolean isAsyncEnabled()

Specified by:

isAsyncEnabled in interface org.apache.flink.table.sources.LookupableTableSource<org.apache.flink.types.Row>

isBounded

public boolean isBounded()

Specified by:

isBounded in interface org.apache.flink.table.sources.StreamTableSource<org.apache.flink.types.Row>

getDataStream

public org.apache.flink.streaming.api.datastream.DataStream<org.apache.flink.types.Row> getDataStream(org.apache.flink.streaming.api.environment.StreamExecutionEnvironment execEnv)

Specified by:

getDataStream in interface org.apache.flink.table.sources.StreamTableSource<org.apache.flink.types.Row>

getInputFormat

protected abstract org.apache.flink.api.common.io.InputFormat<org.apache.flink.types.Row,?> getInputFormat(HBaseTableSchema projectedSchema)

getHBaseTableSchema

@VisibleForTesting
public HBaseTableSchema getHBaseTableSchema()