org.apache.flink.api.java.hadoop.mapred

Class HadoopInputFormatBase<K,V,T>

java.lang.Object

org.apache.flink.api.common.io.RichInputFormat<T,SPITTYPE>

org.apache.flink.api.java.hadoop.common.HadoopInputFormatCommonBase<T,HadoopInputSplit>

org.apache.flink.api.java.hadoop.mapred.HadoopInputFormatBase<K,V,T>

Type Parameters:

K - Type of key

V - Type of value

T - The type itself

All Implemented Interfaces:

Serializable, org.apache.flink.api.common.io.InputFormat<T,HadoopInputSplit>, org.apache.flink.core.io.InputSplitSource<HadoopInputSplit>

Direct Known Subclasses:

HadoopInputFormat

@Internal
public abstract class HadoopInputFormatBase<K,V,T>
extends HadoopInputFormatCommonBase<T,HadoopInputSplit>

Common base for Java and Scala API for using Hadoop input formats with Flink.

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
protected boolean	fetched
protected boolean	hasNext
protected K	key
protected Class<K>	keyClass
protected V	value
protected Class<V>	valueClass

Fields inherited from class org.apache.flink.api.java.hadoop.common.HadoopInputFormatCommonBase

credentials

Constructor Summary

Constructors

Constructor and Description
HadoopInputFormatBase(org.apache.hadoop.mapred.InputFormat<K,V> mapredInputFormat, Class<K> key, Class<V> value, org.apache.hadoop.mapred.JobConf job)

Method Summary

Modifier and Type	Method and Description
void	close()
void	configure(org.apache.flink.configuration.Configuration parameters)
HadoopInputSplit[]	createInputSplits(int minNumSplits)
protected void	fetchNext()
org.apache.flink.core.io.InputSplitAssigner	getInputSplitAssigner(HadoopInputSplit[] inputSplits)
org.apache.hadoop.mapred.JobConf	getJobConf()
org.apache.flink.api.common.io.statistics.BaseStatistics	getStatistics(org.apache.flink.api.common.io.statistics.BaseStatistics cachedStats)
void	open(HadoopInputSplit split)
boolean	reachedEnd()

Methods inherited from class org.apache.flink.api.java.hadoop.common.HadoopInputFormatCommonBase

getCredentialsFromUGI, read, write

Methods inherited from class org.apache.flink.api.common.io.RichInputFormat

closeInputFormat, getRuntimeContext, openInputFormat, setRuntimeContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.api.common.io.InputFormat

nextRecord

Field Detail

keyClass

protected Class<K> keyClass

valueClass

protected Class<V> valueClass

key

protected transient K key

value

protected transient V value

fetched

protected transient boolean fetched

hasNext

protected transient boolean hasNext

Constructor Detail

HadoopInputFormatBase

public HadoopInputFormatBase(org.apache.hadoop.mapred.InputFormat<K,V> mapredInputFormat,
                             Class<K> key,
                             Class<V> value,
                             org.apache.hadoop.mapred.JobConf job)

Method Detail

getJobConf

public org.apache.hadoop.mapred.JobConf getJobConf()

configure

public void configure(org.apache.flink.configuration.Configuration parameters)

getStatistics

public org.apache.flink.api.common.io.statistics.BaseStatistics getStatistics(org.apache.flink.api.common.io.statistics.BaseStatistics cachedStats)
                                                                       throws IOException

Throws:

IOException

createInputSplits

public HadoopInputSplit[] createInputSplits(int minNumSplits)
                                     throws IOException

Throws:

IOException

getInputSplitAssigner

public org.apache.flink.core.io.InputSplitAssigner getInputSplitAssigner(HadoopInputSplit[] inputSplits)

open

public void open(HadoopInputSplit split)
          throws IOException

Throws:

IOException

reachedEnd

public boolean reachedEnd()
                   throws IOException

Throws:

IOException

fetchNext

protected void fetchNext()
                  throws IOException

Throws:

IOException

close

public void close()
           throws IOException

Throws:

IOException