A B C D E F G H I M N O P R S T W

A

add(String, E) - Method in class org.apache.flink.connector.base.source.reader.RecordsBySplits.Builder

Add the record from the given split ID.

add(SourceSplit, E) - Method in class org.apache.flink.connector.base.source.reader.RecordsBySplits.Builder

Add the record from the given source split.

addAll(String, Collection<E>) - Method in class org.apache.flink.connector.base.source.reader.RecordsBySplits.Builder

Add multiple records from the given split ID.

addAll(SourceSplit, Collection<E>) - Method in class org.apache.flink.connector.base.source.reader.RecordsBySplits.Builder

Add multiple records from the given source split.

addFinishedSplit(String) - Method in class org.apache.flink.connector.base.source.reader.RecordsBySplits.Builder

Mark the split with the given ID as finished.

addFinishedSplits(Collection<String>) - Method in class org.apache.flink.connector.base.source.reader.RecordsBySplits.Builder

Mark multiple splits with the given IDs as finished.

addSplits(List<SplitT>) - Method in class org.apache.flink.connector.base.source.reader.fetcher.SingleThreadFetcherManager

addSplits(List<SplitT>) - Method in class org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher

Add splits to the split fetcher.

addSplits(List<SplitT>) - Method in class org.apache.flink.connector.base.source.reader.fetcher.SplitFetcherManager

addSplits(List<SplitT>) - Method in class org.apache.flink.connector.base.source.reader.SourceReaderBase

AVAILABLE - Static variable in class org.apache.flink.connector.base.source.reader.synchronization.FutureCompletingBlockingQueue

A constant future that is complete, indicating availability.

B

build() - Method in class org.apache.flink.connector.base.source.reader.RecordsBySplits.Builder

Builder() - Constructor for class org.apache.flink.connector.base.source.reader.RecordsBySplits.Builder

C

checkErrors() - Method in class org.apache.flink.connector.base.source.reader.fetcher.SplitFetcherManager

close(long) - Method in class org.apache.flink.connector.base.source.reader.fetcher.SplitFetcherManager

Close the split fetcher manager.

close() - Method in class org.apache.flink.connector.base.source.reader.SourceReaderBase

close() - Method in interface org.apache.flink.connector.base.source.reader.splitreader.SplitReader

Close the split reader.

config - Variable in class org.apache.flink.connector.base.source.reader.SourceReaderBase

The raw configurations that may be used by subclasses.

context - Variable in class org.apache.flink.connector.base.source.reader.SourceReaderBase

The context of this source reader.

createSplitFetcher() - Method in class org.apache.flink.connector.base.source.reader.fetcher.SplitFetcherManager

Synchronize method to ensure no fetcher is created after the split fetcher manager has closed.

D

deserializeSplitAssignments(byte[], SimpleVersionedSerializer<SplitT>, Function<Integer, C>) - Static method in class org.apache.flink.connector.base.source.utils.SerdeUtils

Deserialize the given bytes returned by serializeSplitAssignments(Map, SimpleVersionedSerializer).

E

ELEMENT_QUEUE_CAPACITY - Static variable in class org.apache.flink.connector.base.source.reader.SourceReaderOptions

elementQueueCapacity - Variable in class org.apache.flink.connector.base.source.reader.SourceReaderOptions

emitRecord(E, SourceOutput<T>, SplitStateT) - Method in interface org.apache.flink.connector.base.source.reader.RecordEmitter

Process and emit the records to the SourceOutput.

enqueueTask(SplitFetcherTask) - Method in class org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher

F

fetch() - Method in interface org.apache.flink.connector.base.source.reader.splitreader.SplitReader

Fetch elements into the blocking queue for the given splits.

fetchers - Variable in class org.apache.flink.connector.base.source.reader.fetcher.SplitFetcherManager

A map keeping track of all the split fetchers.

finishedSplits() - Method in class org.apache.flink.connector.base.source.reader.RecordsBySplits

finishedSplits() - Method in interface org.apache.flink.connector.base.source.reader.RecordsWithSplitIds

Get the finished splits.

FutureCompletingBlockingQueue<T> - Class in org.apache.flink.connector.base.source.reader.synchronization

A custom implementation of blocking queue in combination with a CompletableFuture that is used in the hand-over of data from a producing thread to a consuming thread.

FutureCompletingBlockingQueue() - Constructor for class org.apache.flink.connector.base.source.reader.synchronization.FutureCompletingBlockingQueue

FutureCompletingBlockingQueue(int) - Constructor for class org.apache.flink.connector.base.source.reader.synchronization.FutureCompletingBlockingQueue

G

getAvailabilityFuture() - Method in class org.apache.flink.connector.base.source.reader.synchronization.FutureCompletingBlockingQueue

Returns the availability future.

getNumAliveFetchers() - Method in class org.apache.flink.connector.base.source.reader.fetcher.SplitFetcherManager

getNumberOfCurrentlyAssignedSplits() - Method in class org.apache.flink.connector.base.source.reader.SourceReaderBase

Gets the number of splits the reads has currently assigned.

getRunningFetcher() - Method in class org.apache.flink.connector.base.source.reader.fetcher.SingleThreadFetcherManager

getSplitReader() - Method in class org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher

H

handleSourceEvents(SourceEvent) - Method in class org.apache.flink.connector.base.source.reader.SourceReaderBase

handleSplitsChanges(SplitsChange<SplitT>) - Method in interface org.apache.flink.connector.base.source.reader.splitreader.SplitReader

Handle the split changes.

I

initializedState(SplitT) - Method in class org.apache.flink.connector.base.source.reader.SourceReaderBase

When new splits are added to the reader.

isAvailable() - Method in class org.apache.flink.connector.base.source.reader.SourceReaderBase

isEmpty() - Method in class org.apache.flink.connector.base.source.reader.synchronization.FutureCompletingBlockingQueue

Checks whether the queue is empty.

M

maybeShutdownFinishedFetchers() - Method in class org.apache.flink.connector.base.source.reader.fetcher.SplitFetcherManager

Check and shutdown the fetchers that have completed their work.

N

nextRecordFromSplit() - Method in class org.apache.flink.connector.base.source.reader.RecordsBySplits

nextRecordFromSplit() - Method in interface org.apache.flink.connector.base.source.reader.RecordsWithSplitIds

Gets the next record from the current split.

nextSplit() - Method in class org.apache.flink.connector.base.source.reader.RecordsBySplits

nextSplit() - Method in interface org.apache.flink.connector.base.source.reader.RecordsWithSplitIds

Moves to the next split.

notifyAvailable() - Method in class org.apache.flink.connector.base.source.reader.synchronization.FutureCompletingBlockingQueue

Makes sure the availability future is complete, if it is not complete already.

notifyNoMoreSplits() - Method in class org.apache.flink.connector.base.source.reader.SourceReaderBase

O

onSplitFinished(Map<String, SplitStateT>) - Method in class org.apache.flink.connector.base.source.reader.SourceReaderBase

Handles the finished splits to clean the state if needed.

options - Variable in class org.apache.flink.connector.base.source.reader.SourceReaderBase

The configuration for the reader.

org.apache.flink.connector.base.source.reader - package org.apache.flink.connector.base.source.reader

org.apache.flink.connector.base.source.reader.fetcher - package org.apache.flink.connector.base.source.reader.fetcher

org.apache.flink.connector.base.source.reader.splitreader - package org.apache.flink.connector.base.source.reader.splitreader

org.apache.flink.connector.base.source.reader.synchronization - package org.apache.flink.connector.base.source.reader.synchronization

org.apache.flink.connector.base.source.utils - package org.apache.flink.connector.base.source.utils

P

peek() - Method in class org.apache.flink.connector.base.source.reader.synchronization.FutureCompletingBlockingQueue

Get the first element from the queue without removing it.

poll() - Method in class org.apache.flink.connector.base.source.reader.synchronization.FutureCompletingBlockingQueue

Get and remove the first element from the queue.

pollNext(ReaderOutput<T>) - Method in class org.apache.flink.connector.base.source.reader.SourceReaderBase

put(int, T) - Method in class org.apache.flink.connector.base.source.reader.synchronization.FutureCompletingBlockingQueue

Put an element into the queue.

R

RecordEmitter<E,T,SplitStateT> - Interface in org.apache.flink.connector.base.source.reader

Emit a record to the downstream.

recordEmitter - Variable in class org.apache.flink.connector.base.source.reader.SourceReaderBase

The record emitter to handle the records read by the SplitReaders.

RecordsBySplits<E> - Class in org.apache.flink.connector.base.source.reader

An implementation of RecordsWithSplitIds to host all the records by splits.

RecordsBySplits(Map<String, Collection<E>>, Set<String>) - Constructor for class org.apache.flink.connector.base.source.reader.RecordsBySplits

RecordsBySplits.Builder<E> - Class in org.apache.flink.connector.base.source.reader

A utility builder to collect records in individual calls, rather than put a finished collection in the RecordsBySplits.RecordsBySplits(Map, Set) constructor.

RecordsWithSplitIds<E> - Interface in org.apache.flink.connector.base.source.reader

An interface for the elements passed from the fetchers to the source reader.

recycle() - Method in interface org.apache.flink.connector.base.source.reader.RecordsWithSplitIds

This method is called when all records from this batch have been emitted.

remainingCapacity() - Method in class org.apache.flink.connector.base.source.reader.synchronization.FutureCompletingBlockingQueue

Checks the remaining capacity in the queue.

run() - Method in class org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher

run() - Method in interface org.apache.flink.connector.base.source.reader.fetcher.SplitFetcherTask

Run the logic.

S

SerdeUtils - Class in org.apache.flink.connector.base.source.utils

A util class with some helper method for serde in the sources.

serializeSplitAssignments(Map<Integer, C>, SimpleVersionedSerializer<SplitT>) - Static method in class org.apache.flink.connector.base.source.utils.SerdeUtils

Serialize a mapping from subtask ids to lists of assigned splits.

shutdown() - Method in class org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher

Shutdown the split fetcher.

SingleThreadFetcherManager<E,SplitT extends org.apache.flink.api.connector.source.SourceSplit> - Class in org.apache.flink.connector.base.source.reader.fetcher

A Fetcher Manager with a single fetching thread (I/O thread) that handles all splits concurrently.

SingleThreadFetcherManager(FutureCompletingBlockingQueue<RecordsWithSplitIds<E>>, Supplier<SplitReader<E, SplitT>>) - Constructor for class org.apache.flink.connector.base.source.reader.fetcher.SingleThreadFetcherManager

Creates a new SplitFetcherManager with a single I/O threads.

SingleThreadMultiplexSourceReaderBase<E,T,SplitT extends org.apache.flink.api.connector.source.SourceSplit,SplitStateT> - Class in org.apache.flink.connector.base.source.reader

A base for SourceReaders that read splits with one thread using one SplitReader.

SingleThreadMultiplexSourceReaderBase(Supplier<SplitReader<E, SplitT>>, RecordEmitter<E, T, SplitStateT>, Configuration, SourceReaderContext) - Constructor for class org.apache.flink.connector.base.source.reader.SingleThreadMultiplexSourceReaderBase

The primary constructor for the source reader.

SingleThreadMultiplexSourceReaderBase(FutureCompletingBlockingQueue<RecordsWithSplitIds<E>>, Supplier<SplitReader<E, SplitT>>, RecordEmitter<E, T, SplitStateT>, Configuration, SourceReaderContext) - Constructor for class org.apache.flink.connector.base.source.reader.SingleThreadMultiplexSourceReaderBase

This constructor behaves like SingleThreadMultiplexSourceReaderBase.SingleThreadMultiplexSourceReaderBase(Supplier, RecordEmitter, Configuration, SourceReaderContext), but accepts a specific FutureCompletingBlockingQueue.

SingleThreadMultiplexSourceReaderBase(FutureCompletingBlockingQueue<RecordsWithSplitIds<E>>, SingleThreadFetcherManager<E, SplitT>, RecordEmitter<E, T, SplitStateT>, Configuration, SourceReaderContext) - Constructor for class org.apache.flink.connector.base.source.reader.SingleThreadMultiplexSourceReaderBase

This constructor behaves like SingleThreadMultiplexSourceReaderBase.SingleThreadMultiplexSourceReaderBase(Supplier, RecordEmitter, Configuration, SourceReaderContext), but accepts a specific FutureCompletingBlockingQueue and SingleThreadFetcherManager.

size() - Method in class org.apache.flink.connector.base.source.reader.synchronization.FutureCompletingBlockingQueue

Gets the size of the queue.

snapshotState(long) - Method in class org.apache.flink.connector.base.source.reader.SourceReaderBase

SOURCE_READER_CLOSE_TIMEOUT - Static variable in class org.apache.flink.connector.base.source.reader.SourceReaderOptions

SourceReaderBase<E,T,SplitT extends org.apache.flink.api.connector.source.SourceSplit,SplitStateT> - Class in org.apache.flink.connector.base.source.reader

An abstract implementation of SourceReader which provides some sychronization between the mail box main thread and the SourceReader internal threads.

SourceReaderBase(FutureCompletingBlockingQueue<RecordsWithSplitIds<E>>, SplitFetcherManager<E, SplitT>, RecordEmitter<E, T, SplitStateT>, Configuration, SourceReaderContext) - Constructor for class org.apache.flink.connector.base.source.reader.SourceReaderBase

sourceReaderCloseTimeout - Variable in class org.apache.flink.connector.base.source.reader.SourceReaderOptions

SourceReaderOptions - Class in org.apache.flink.connector.base.source.reader

The options tht can be set for the SourceReaderBase.

SourceReaderOptions(Configuration) - Constructor for class org.apache.flink.connector.base.source.reader.SourceReaderOptions

SplitFetcher<E,SplitT extends org.apache.flink.api.connector.source.SourceSplit> - Class in org.apache.flink.connector.base.source.reader.fetcher

The internal fetcher runnable responsible for polling message from the external system.

SplitFetcherManager<E,SplitT extends org.apache.flink.api.connector.source.SourceSplit> - Class in org.apache.flink.connector.base.source.reader.fetcher

A class responsible for starting the SplitFetcher and manage the life cycles of them.

SplitFetcherManager(FutureCompletingBlockingQueue<RecordsWithSplitIds<E>>, Supplier<SplitReader<E, SplitT>>) - Constructor for class org.apache.flink.connector.base.source.reader.fetcher.SplitFetcherManager

Create a split fetcher manager.

splitFetcherManager - Variable in class org.apache.flink.connector.base.source.reader.SourceReaderBase

The split fetcher manager to run split fetchers.

SplitFetcherTask - Interface in org.apache.flink.connector.base.source.reader.fetcher

An interface similar to Runnable but allows throwing exceptions and wakeup.

SplitReader<E,SplitT extends org.apache.flink.api.connector.source.SourceSplit> - Interface in org.apache.flink.connector.base.source.reader.splitreader

An interface used to read from splits.

splits() - Method in class org.apache.flink.connector.base.source.reader.splitreader.SplitsChange

SplitsAddition<SplitT> - Class in org.apache.flink.connector.base.source.reader.splitreader

A change to add splits.

SplitsAddition(List<SplitT>) - Constructor for class org.apache.flink.connector.base.source.reader.splitreader.SplitsAddition

SplitsChange<SplitT> - Class in org.apache.flink.connector.base.source.reader.splitreader

An abstract class to host splits change.

start() - Method in class org.apache.flink.connector.base.source.reader.SourceReaderBase

startFetcher(SplitFetcher<E, SplitT>) - Method in class org.apache.flink.connector.base.source.reader.fetcher.SplitFetcherManager

T

take() - Method in class org.apache.flink.connector.base.source.reader.synchronization.FutureCompletingBlockingQueue

Warning: This is a dangerous method and should only be used for testing convenience.

toSplitType(String, SplitStateT) - Method in class org.apache.flink.connector.base.source.reader.SourceReaderBase

Convert a mutable SplitStateT to immutable SplitT.

toString() - Method in class org.apache.flink.connector.base.source.reader.splitreader.SplitsAddition

W

wakeUp() - Method in interface org.apache.flink.connector.base.source.reader.fetcher.SplitFetcherTask

Wake up the running thread.

wakeUp() - Method in interface org.apache.flink.connector.base.source.reader.splitreader.SplitReader

Wake up the split reader in case the fetcher thread is blocking in SplitReader.fetch().

wakeUpPuttingThread(int) - Method in class org.apache.flink.connector.base.source.reader.synchronization.FutureCompletingBlockingQueue

Gracefully wakes up the thread with the given threadIndex if it is blocked in adding an element.

A B C D E F G H I M N O P R S T W