org.apache.flink.streaming.api.operators

类 SourceOperator<OUT,SplitT extends org.apache.flink.api.connector.source.SourceSplit>

java.lang.Object

org.apache.flink.streaming.api.operators.AbstractStreamOperator<OUT>

org.apache.flink.streaming.api.operators.SourceOperator<OUT,SplitT>

类型参数:

OUT - The output type of the operator.

所有已实现的接口:

Serializable, org.apache.flink.runtime.io.AvailabilityProvider, org.apache.flink.runtime.operators.coordination.OperatorEventHandler, org.apache.flink.runtime.state.CheckpointListener, KeyContext, SetupableStreamOperator<OUT>, StreamOperator<OUT>, StreamOperatorStateHandler.CheckpointedStreamOperator, PushingAsyncDataInput<OUT>, org.apache.flink.util.Disposable

@Internal
public class SourceOperator<OUT,SplitT extends org.apache.flink.api.connector.source.SourceSplit>
extends AbstractStreamOperator<OUT>
implements org.apache.flink.runtime.operators.coordination.OperatorEventHandler, PushingAsyncDataInput<OUT>

Base source operator only used for integrating the source reader which is proposed by FLIP-27. It implements the interface of PushingAsyncDataInput for naturally compatible with one input processing in runtime stack.

Important Note on Serialization: The SourceOperator inherits the Serializable interface from the StreamOperator, but is in fact NOT serializable. The operator must only be instantiates in the StreamTask from its factory.

另请参阅:

序列化表格

嵌套类概要

从接口继承的嵌套类/接口 org.apache.flink.streaming.runtime.io.PushingAsyncDataInput

PushingAsyncDataInput.DataOutput<T>

从接口继承的嵌套类/接口 org.apache.flink.runtime.io.AvailabilityProvider

org.apache.flink.runtime.io.AvailabilityProvider.AvailabilityHelper

字段概要

从类继承的字段 org.apache.flink.streaming.api.operators.AbstractStreamOperator

chainingStrategy, config, latencyStats, LOG, metrics, output, processingTimeService

从接口继承的字段 org.apache.flink.runtime.io.AvailabilityProvider

AVAILABLE

构造器概要

构造器

构造器和说明

SourceOperator(org.apache.flink.util.function.FunctionWithException<org.apache.flink.api.connector.source.SourceReaderContext,org.apache.flink.api.connector.source.SourceReader<OUT,SplitT>,Exception> readerFactory, org.apache.flink.runtime.operators.coordination.OperatorEventGateway operatorEventGateway, org.apache.flink.core.io.SimpleVersionedSerializer<SplitT> splitSerializer, org.apache.flink.api.common.eventtime.WatermarkStrategy<OUT> watermarkStrategy, ProcessingTimeService timeService, org.apache.flink.configuration.Configuration configuration, String localHostname)

方法概要

限定符和类型	方法和说明
void	close() This method is called after all records have been added to the operators via the methods OneInputStreamOperator.processElement(StreamRecord), or TwoInputStreamOperator.processElement1(StreamRecord) and TwoInputStreamOperator.processElement2(StreamRecord).
void	dispose() This method is called at the very end of the operator's life, both in the case of a successful completion of the operation, and in the case of a failure and canceling.
org.apache.flink.core.io.InputStatus	emitNext(PushingAsyncDataInput.DataOutput<OUT> output) Pushes the next element to the output from current data input, and returns the input status to indicate whether there are more available data in current input.
CompletableFuture<?>	getAvailableFuture()
org.apache.flink.api.connector.source.SourceReader<OUT,SplitT>	getSourceReader()
void	handleOperatorEvent(org.apache.flink.runtime.operators.coordination.OperatorEvent event)
void	initializeState(org.apache.flink.runtime.state.StateInitializationContext context) Stream operators with state which can be restored need to override this hook method.
void	initReader() Initializes the reader.
void	notifyCheckpointAborted(long checkpointId)
void	notifyCheckpointComplete(long checkpointId)
void	open() This method is called immediately before any elements are processed, it should contain the operator's initialization logic, e.g. state initialization.
void	snapshotState(org.apache.flink.runtime.state.StateSnapshotContext context) Stream operators with state, which want to participate in a snapshot need to override this hook method.

从类继承的方法 org.apache.flink.streaming.api.operators.AbstractStreamOperator

getChainingStrategy, getContainingTask, getCurrentKey, getExecutionConfig, getInternalTimerService, getKeyedStateBackend, getKeyedStateStore, getMetricGroup, getOperatorConfig, getOperatorID, getOperatorName, getOperatorStateBackend, getOrCreateKeyedState, getPartitionedState, getPartitionedState, getProcessingTimeService, getRuntimeContext, getTimeServiceManager, getUserCodeClassloader, initializeState, isUsingCustomRawKeyedState, numEventTimeTimers, numProcessingTimeTimers, prepareSnapshotPreBarrier, processLatencyMarker, processLatencyMarker1, processLatencyMarker2, processWatermark, processWatermark1, processWatermark2, reportOrForwardLatencyMarker, setChainingStrategy, setCurrentKey, setKeyContextElement1, setKeyContextElement2, setProcessingTimeService, setup, snapshotState

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

从接口继承的方法 org.apache.flink.runtime.io.AvailabilityProvider

isApproximatelyAvailable, isAvailable

构造器详细资料

SourceOperator

public SourceOperator(org.apache.flink.util.function.FunctionWithException<org.apache.flink.api.connector.source.SourceReaderContext,org.apache.flink.api.connector.source.SourceReader<OUT,SplitT>,Exception> readerFactory,
                      org.apache.flink.runtime.operators.coordination.OperatorEventGateway operatorEventGateway,
                      org.apache.flink.core.io.SimpleVersionedSerializer<SplitT> splitSerializer,
                      org.apache.flink.api.common.eventtime.WatermarkStrategy<OUT> watermarkStrategy,
                      ProcessingTimeService timeService,
                      org.apache.flink.configuration.Configuration configuration,
                      String localHostname)

方法详细资料

initReader

public void initReader()
                throws Exception

Initializes the reader. The code from this method should ideally happen in the constructor or in the operator factory even. It has to happen here at a slightly later stage, because of the lazy metric initialization.

Calling this method explicitly is an optional way to have the reader initialization a bit earlier than in open(), as needed by the SourceOperatorStreamTask

This code should move to the constructor once the metric groups are available at task setup time.

抛出:

Exception

open

public void open()
          throws Exception

从类复制的说明: AbstractStreamOperator

This method is called immediately before any elements are processed, it should contain the operator's initialization logic, e.g. state initialization.

The default implementation does nothing.

指定者:

open 在接口中 StreamOperator<OUT>

覆盖:

open 在类中 AbstractStreamOperator<OUT>

抛出:

Exception - An exception in this method causes the operator to fail.

close

public void close()
           throws Exception

从类复制的说明: AbstractStreamOperator

This method is called after all records have been added to the operators via the methods OneInputStreamOperator.processElement(StreamRecord), or TwoInputStreamOperator.processElement1(StreamRecord) and TwoInputStreamOperator.processElement2(StreamRecord).

The method is expected to flush all remaining buffered data. Exceptions during this flushing of buffered should be propagated, in order to cause the operation to be recognized asa failed, because the last data items are not processed properly.

指定者:

close 在接口中 StreamOperator<OUT>

覆盖:

close 在类中 AbstractStreamOperator<OUT>

抛出:

Exception - An exception in this method causes the operator to fail.

dispose

public void dispose()
             throws Exception

从类复制的说明: AbstractStreamOperator

This method is called at the very end of the operator's life, both in the case of a successful completion of the operation, and in the case of a failure and canceling.

This method is expected to make a thorough effort to release all resources that the operator has acquired.

指定者:

dispose 在接口中 StreamOperator<OUT>

指定者:

dispose 在接口中 org.apache.flink.util.Disposable

覆盖:

dispose 在类中 AbstractStreamOperator<OUT>

抛出:

Exception

emitNext

public org.apache.flink.core.io.InputStatus emitNext(PushingAsyncDataInput.DataOutput<OUT> output)
                                              throws Exception

从接口复制的说明: PushingAsyncDataInput

Pushes the next element to the output from current data input, and returns the input status to indicate whether there are more available data in current input.

This method should be non blocking.

指定者:

emitNext 在接口中 PushingAsyncDataInput<OUT>

抛出:

Exception

snapshotState

public void snapshotState(org.apache.flink.runtime.state.StateSnapshotContext context)
                   throws Exception

从类复制的说明: AbstractStreamOperator

Stream operators with state, which want to participate in a snapshot need to override this hook method.

指定者:

snapshotState 在接口中 StreamOperatorStateHandler.CheckpointedStreamOperator

覆盖:

snapshotState 在类中 AbstractStreamOperator<OUT>

参数:

context - context that provides information and means required for taking a snapshot

抛出:

Exception

getAvailableFuture

public CompletableFuture<?> getAvailableFuture()

指定者:

getAvailableFuture 在接口中 org.apache.flink.runtime.io.AvailabilityProvider

initializeState

public void initializeState(org.apache.flink.runtime.state.StateInitializationContext context)
                     throws Exception

从类复制的说明: AbstractStreamOperator

Stream operators with state which can be restored need to override this hook method.

指定者:

initializeState 在接口中 StreamOperatorStateHandler.CheckpointedStreamOperator

覆盖:

initializeState 在类中 AbstractStreamOperator<OUT>

参数:

context - context that allows to register different states.

抛出:

Exception

notifyCheckpointComplete

public void notifyCheckpointComplete(long checkpointId)
                              throws Exception

指定者:

notifyCheckpointComplete 在接口中 org.apache.flink.runtime.state.CheckpointListener

覆盖:

notifyCheckpointComplete 在类中 AbstractStreamOperator<OUT>

抛出:

Exception

notifyCheckpointAborted

public void notifyCheckpointAborted(long checkpointId)
                             throws Exception

指定者:

notifyCheckpointAborted 在接口中 org.apache.flink.runtime.state.CheckpointListener

覆盖:

notifyCheckpointAborted 在类中 AbstractStreamOperator<OUT>

抛出:

Exception

handleOperatorEvent

public void handleOperatorEvent(org.apache.flink.runtime.operators.coordination.OperatorEvent event)

指定者:

handleOperatorEvent 在接口中 org.apache.flink.runtime.operators.coordination.OperatorEventHandler

getSourceReader

@VisibleForTesting
public org.apache.flink.api.connector.source.SourceReader<OUT,SplitT> getSourceReader()