org.apache.flink.batch.connectors.pulsar

Class PulsarInputFormat<T>

java.lang.Object

org.apache.flink.api.common.io.RichInputFormat<T,PulsarInputSplit>

org.apache.flink.batch.connectors.pulsar.PulsarInputFormat<T>

Type Parameters:

T - the type of record.

All Implemented Interfaces:

Serializable, org.apache.flink.api.common.io.InputFormat<T,PulsarInputSplit>, org.apache.flink.api.java.typeutils.ResultTypeQueryable<T>, org.apache.flink.core.io.InputSplitSource<PulsarInputSplit>

public class PulsarInputFormat<T>
extends org.apache.flink.api.common.io.RichInputFormat<T,PulsarInputSplit>
implements org.apache.flink.api.java.typeutils.ResultTypeQueryable<T>

Read Pulsar data in batch mode. Bypass broker and read segments directly for efficiency.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
PulsarInputFormat(ConnectorConfig connectorConfig, org.apache.flink.api.common.serialization.DeserializationSchema<T> deserializer)

Method Summary

Modifier and Type	Method and Description
void	close()
void	configure(org.apache.flink.configuration.Configuration parameters)
PulsarInputSplit[]	createInputSplits(int minNumSplits)
protected PulsarInputSplit	genSplit(int index, List<InputLedger> ledgers)
org.apache.flink.core.io.InputSplitAssigner	getInputSplitAssigner(PulsarInputSplit[] inputSplits)
org.apache.flink.api.common.typeinfo.TypeInformation<T>	getProducedType()
org.apache.flink.api.common.io.statistics.BaseStatistics	getStatistics(org.apache.flink.api.common.io.statistics.BaseStatistics cachedStatistics)
T	nextRecord(T reuse)
void	open(PulsarInputSplit split)
boolean	reachedEnd()

Methods inherited from class org.apache.flink.api.common.io.RichInputFormat

closeInputFormat, getRuntimeContext, openInputFormat, setRuntimeContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

PulsarInputFormat

public PulsarInputFormat(ConnectorConfig connectorConfig,
                         org.apache.flink.api.common.serialization.DeserializationSchema<T> deserializer)

Method Detail

configure

public void configure(org.apache.flink.configuration.Configuration parameters)

Specified by:

configure in interface org.apache.flink.api.common.io.InputFormat<T,PulsarInputSplit>

getStatistics

public org.apache.flink.api.common.io.statistics.BaseStatistics getStatistics(org.apache.flink.api.common.io.statistics.BaseStatistics cachedStatistics)
                                                                       throws IOException

Specified by:

getStatistics in interface org.apache.flink.api.common.io.InputFormat<T,PulsarInputSplit>

Throws:

IOException

createInputSplits

public PulsarInputSplit[] createInputSplits(int minNumSplits)
                                     throws IOException

Specified by:

createInputSplits in interface org.apache.flink.api.common.io.InputFormat<T,PulsarInputSplit>

Specified by:

createInputSplits in interface org.apache.flink.core.io.InputSplitSource<PulsarInputSplit>

Throws:

IOException

genSplit

protected PulsarInputSplit genSplit(int index,
                                    List<InputLedger> ledgers)

getInputSplitAssigner

public org.apache.flink.core.io.InputSplitAssigner getInputSplitAssigner(PulsarInputSplit[] inputSplits)

Specified by:

getInputSplitAssigner in interface org.apache.flink.api.common.io.InputFormat<T,PulsarInputSplit>

Specified by:

getInputSplitAssigner in interface org.apache.flink.core.io.InputSplitSource<PulsarInputSplit>

open

public void open(PulsarInputSplit split)
          throws IOException

Specified by:

open in interface org.apache.flink.api.common.io.InputFormat<T,PulsarInputSplit>

Throws:

IOException

reachedEnd

public boolean reachedEnd()
                   throws IOException

Specified by:

reachedEnd in interface org.apache.flink.api.common.io.InputFormat<T,PulsarInputSplit>

Throws:

IOException

nextRecord

public T nextRecord(T reuse)
             throws IOException

Specified by:

nextRecord in interface org.apache.flink.api.common.io.InputFormat<T,PulsarInputSplit>

Throws:

IOException

close

public void close()
           throws IOException

Specified by:

close in interface org.apache.flink.api.common.io.InputFormat<T,PulsarInputSplit>

Throws:

IOException

getProducedType

public org.apache.flink.api.common.typeinfo.TypeInformation<T> getProducedType()

Specified by:

getProducedType in interface org.apache.flink.api.java.typeutils.ResultTypeQueryable<T>