org.apache.flink.connector.hbase.source

类 HBaseInputFormat<T extends org.apache.flink.api.java.tuple.Tuple>

java.lang.Object

org.apache.flink.api.common.io.RichInputFormat<T,org.apache.flink.connector.hbase.source.TableInputSplit>

org.apache.flink.connector.hbase.source.HBaseInputFormat<T>

所有已实现的接口:

Serializable, org.apache.flink.api.common.io.InputFormat<T,org.apache.flink.connector.hbase.source.TableInputSplit>, org.apache.flink.core.io.InputSplitSource<org.apache.flink.connector.hbase.source.TableInputSplit>

直接已知子类:

TableInputFormat

@Experimental
public abstract class HBaseInputFormat<T extends org.apache.flink.api.java.tuple.Tuple>
extends org.apache.flink.api.common.io.RichInputFormat<T,org.apache.flink.connector.hbase.source.TableInputSplit>

InputFormat subclass that wraps the access for HTables.

另请参阅:

序列化表格

字段概要

字段

限定符和类型	字段和说明
protected byte[]	currentRow
protected boolean	endReached
protected static org.slf4j.Logger	LOG
protected org.apache.hadoop.hbase.client.ResultScanner	resultScanner HBase iterator wrapper.
protected org.apache.hadoop.hbase.client.Scan	scan
protected long	scannedRows
protected byte[]	serializedConfig
protected org.apache.hadoop.hbase.client.HTable	table

构造器概要

构造器

构造器和说明
HBaseInputFormat(org.apache.hadoop.conf.Configuration hConf) Constructs a InputFormat with hbase configuration to read data from hbase.

方法概要

限定符和类型	方法和说明
void	close()
void	closeInputFormat()
void	configure(org.apache.flink.configuration.Configuration parameters) Creates a Scan object and opens the HTable connection.
org.apache.flink.connector.hbase.source.TableInputSplit[]	createInputSplits(int minNumSplits)
protected org.apache.hadoop.conf.Configuration	getHadoopConfiguration()
org.apache.flink.core.io.InputSplitAssigner	getInputSplitAssigner(org.apache.flink.connector.hbase.source.TableInputSplit[] inputSplits)
protected abstract org.apache.hadoop.hbase.client.Scan	getScanner() Returns an instance of Scan that retrieves the required subset of records from the HBase table.
org.apache.flink.api.common.io.statistics.BaseStatistics	getStatistics(org.apache.flink.api.common.io.statistics.BaseStatistics cachedStatistics)
protected abstract String	getTableName() What table is to be read.
protected boolean	includeRegionInScan(byte[] startKey, byte[] endKey) Test if the given region is to be included in the scan while splitting the regions of a table.
protected T	mapResultToOutType(org.apache.hadoop.hbase.client.Result r) HBase returns an instance of Result.
protected abstract T	mapResultToTuple(org.apache.hadoop.hbase.client.Result r) The output from HBase is always an instance of Result.
T	nextRecord(T reuse)
void	open(org.apache.flink.connector.hbase.source.TableInputSplit split)
boolean	reachedEnd()

从类继承的方法 org.apache.flink.api.common.io.RichInputFormat

getRuntimeContext, openInputFormat, setRuntimeContext

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

字段详细资料

LOG

protected static final org.slf4j.Logger LOG

endReached

protected boolean endReached

table

protected transient org.apache.hadoop.hbase.client.HTable table

scan

protected transient org.apache.hadoop.hbase.client.Scan scan

resultScanner

protected org.apache.hadoop.hbase.client.ResultScanner resultScanner

HBase iterator wrapper.

currentRow

protected byte[] currentRow

scannedRows

protected long scannedRows

serializedConfig

protected byte[] serializedConfig

构造器详细资料

HBaseInputFormat

public HBaseInputFormat(org.apache.hadoop.conf.Configuration hConf)

Constructs a InputFormat with hbase configuration to read data from hbase.

参数:

hConf - The configuration that connect to hbase. At least hbase.zookeeper.quorum and zookeeper.znode.parent need to be set.

方法详细资料

getScanner

protected abstract org.apache.hadoop.hbase.client.Scan getScanner()

Returns an instance of Scan that retrieves the required subset of records from the HBase table.

返回:

The appropriate instance of Scan for this usecase.

getTableName

protected abstract String getTableName()

What table is to be read. Per instance of a TableInputFormat derivative only a single tablename is possible.

返回:

The name of the table

mapResultToTuple

protected abstract T mapResultToTuple(org.apache.hadoop.hbase.client.Result r)

The output from HBase is always an instance of Result. This method is to copy the data in the Result instance into the required Tuple

参数:

r - The Result instance from HBase that needs to be converted

返回:

The appropriate instance of Tuple that contains the needed information.

configure

public void configure(org.apache.flink.configuration.Configuration parameters)

Creates a Scan object and opens the HTable connection. These are opened here because they are needed in the createInputSplits which is called before the openInputFormat method. So the connection is opened in configure(Configuration) and closed in closeInputFormat().

指定者:

configure 在接口中 org.apache.flink.api.common.io.InputFormat<T extends org.apache.flink.api.java.tuple.Tuple,org.apache.flink.connector.hbase.source.TableInputSplit>

参数:

parameters - The configuration that is to be used

另请参阅:

Configuration

mapResultToOutType

protected T mapResultToOutType(org.apache.hadoop.hbase.client.Result r)

HBase returns an instance of Result.

This method maps the returned Result instance into the output type T.

参数:

r - The Result instance from HBase that needs to be converted

返回:

The appropriate instance of T that contains the data of Result.

getHadoopConfiguration

protected org.apache.hadoop.conf.Configuration getHadoopConfiguration()

open

public void open(org.apache.flink.connector.hbase.source.TableInputSplit split)
          throws IOException

抛出:

IOException

nextRecord

public T nextRecord(T reuse)
             throws IOException

抛出:

IOException

reachedEnd

public boolean reachedEnd()
                   throws IOException

抛出:

IOException

close

public void close()
           throws IOException

抛出:

IOException

closeInputFormat

public void closeInputFormat()
                      throws IOException

覆盖:

closeInputFormat 在类中 org.apache.flink.api.common.io.RichInputFormat<T,org.apache.flink.connector.hbase.source.TableInputSplit>

抛出:

IOException

createInputSplits

public org.apache.flink.connector.hbase.source.TableInputSplit[] createInputSplits(int minNumSplits)
                                                                            throws IOException

抛出:

IOException

includeRegionInScan

protected boolean includeRegionInScan(byte[] startKey,
                                      byte[] endKey)

Test if the given region is to be included in the scan while splitting the regions of a table.

参数:

startKey - Start key of the region

endKey - End key of the region

返回:

true, if this region needs to be included as part of the input (default).

getInputSplitAssigner

public org.apache.flink.core.io.InputSplitAssigner getInputSplitAssigner(org.apache.flink.connector.hbase.source.TableInputSplit[] inputSplits)

getStatistics

public org.apache.flink.api.common.io.statistics.BaseStatistics getStatistics(org.apache.flink.api.common.io.statistics.BaseStatistics cachedStatistics)