org.apache.flink.connector.kafka.source.reader

Class KafkaSourceReader<T>

java.lang.Object

org.apache.flink.connector.base.source.reader.SourceReaderBase<E,T,SplitT,SplitStateT>

org.apache.flink.connector.base.source.reader.SingleThreadMultiplexSourceReaderBase<org.apache.flink.api.java.tuple.Tuple3<T,Long,Long>,T,KafkaPartitionSplit,KafkaPartitionSplitState>

org.apache.flink.connector.kafka.source.reader.KafkaSourceReader<T>

All Implemented Interfaces:

AutoCloseable, org.apache.flink.api.common.state.CheckpointListener, org.apache.flink.api.connector.source.SourceReader<T,KafkaPartitionSplit>

public class KafkaSourceReader<T>
extends org.apache.flink.connector.base.source.reader.SingleThreadMultiplexSourceReaderBase<org.apache.flink.api.java.tuple.Tuple3<T,Long,Long>,T,KafkaPartitionSplit,KafkaPartitionSplitState>

The source reader for Kafka partitions.

Field Summary

Fields inherited from class org.apache.flink.connector.base.source.reader.SourceReaderBase

config, context, options, recordEmitter, splitFetcherManager

Constructor Summary

Constructors

Constructor and Description

KafkaSourceReader(org.apache.flink.connector.base.source.reader.synchronization.FutureCompletingBlockingQueue<org.apache.flink.connector.base.source.reader.RecordsWithSplitIds<org.apache.flink.api.java.tuple.Tuple3<T,Long,Long>>> elementsQueue, KafkaSourceFetcherManager<T> kafkaSourceFetcherManager, org.apache.flink.connector.base.source.reader.RecordEmitter<org.apache.flink.api.java.tuple.Tuple3<T,Long,Long>,T,KafkaPartitionSplitState> recordEmitter, org.apache.flink.configuration.Configuration config, org.apache.flink.api.connector.source.SourceReaderContext context, KafkaSourceReaderMetrics kafkaSourceReaderMetrics)

Method Summary

Modifier and Type	Method and Description
protected KafkaPartitionSplitState	initializedState(KafkaPartitionSplit split)
void	notifyCheckpointComplete(long checkpointId)
protected void	onSplitFinished(Map<String,KafkaPartitionSplitState> finishedSplitIds)
List<KafkaPartitionSplit>	snapshotState(long checkpointId)
protected KafkaPartitionSplit	toSplitType(String splitId, KafkaPartitionSplitState splitState)

Methods inherited from class org.apache.flink.connector.base.source.reader.SourceReaderBase

addSplits, close, getNumberOfCurrentlyAssignedSplits, handleSourceEvents, isAvailable, notifyNoMoreSplits, pollNext, start

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.api.common.state.CheckpointListener

notifyCheckpointAborted

Constructor Detail

KafkaSourceReader

public KafkaSourceReader(org.apache.flink.connector.base.source.reader.synchronization.FutureCompletingBlockingQueue<org.apache.flink.connector.base.source.reader.RecordsWithSplitIds<org.apache.flink.api.java.tuple.Tuple3<T,Long,Long>>> elementsQueue,
                         KafkaSourceFetcherManager<T> kafkaSourceFetcherManager,
                         org.apache.flink.connector.base.source.reader.RecordEmitter<org.apache.flink.api.java.tuple.Tuple3<T,Long,Long>,T,KafkaPartitionSplitState> recordEmitter,
                         org.apache.flink.configuration.Configuration config,
                         org.apache.flink.api.connector.source.SourceReaderContext context,
                         KafkaSourceReaderMetrics kafkaSourceReaderMetrics)

Method Detail

onSplitFinished

protected void onSplitFinished(Map<String,KafkaPartitionSplitState> finishedSplitIds)

Specified by:

onSplitFinished in class org.apache.flink.connector.base.source.reader.SourceReaderBase<org.apache.flink.api.java.tuple.Tuple3<T,Long,Long>,T,KafkaPartitionSplit,KafkaPartitionSplitState>

snapshotState

public List<KafkaPartitionSplit> snapshotState(long checkpointId)

Specified by:

snapshotState in interface org.apache.flink.api.connector.source.SourceReader<T,KafkaPartitionSplit>

Overrides:

snapshotState in class org.apache.flink.connector.base.source.reader.SourceReaderBase<org.apache.flink.api.java.tuple.Tuple3<T,Long,Long>,T,KafkaPartitionSplit,KafkaPartitionSplitState>

notifyCheckpointComplete

public void notifyCheckpointComplete(long checkpointId)
                              throws Exception

Throws:

Exception

initializedState

protected KafkaPartitionSplitState initializedState(KafkaPartitionSplit split)

Specified by:

initializedState in class org.apache.flink.connector.base.source.reader.SourceReaderBase<org.apache.flink.api.java.tuple.Tuple3<T,Long,Long>,T,KafkaPartitionSplit,KafkaPartitionSplitState>

toSplitType

protected KafkaPartitionSplit toSplitType(String splitId,
                                          KafkaPartitionSplitState splitState)

Specified by:

toSplitType in class org.apache.flink.connector.base.source.reader.SourceReaderBase<org.apache.flink.api.java.tuple.Tuple3<T,Long,Long>,T,KafkaPartitionSplit,KafkaPartitionSplitState>