IN - Input type for the operator.OUT - Output type for the operator.@Internal public class AsyncWaitOperator<IN,OUT> extends AbstractUdfStreamOperator<OUT,AsyncFunction<IN,OUT>> implements OneInputStreamOperator<IN,OUT>, BoundedOneInput
AsyncWaitOperator allows to asynchronously process incoming stream records. For that
the operator creates an ResultFuture which is passed to an AsyncFunction. Within
the async function, the user can complete the async collector arbitrarily. Once the async
collector has been completed, the result is emitted by the operator's emitter to downstream
operators.
The operator offers different output modes depending on the chosen AsyncDataStream.OutputMode. In
order to give exactly once processing guarantees, the operator stores all currently in-flight
StreamElement in it's operator state. Upon recovery the recorded set of stream elements
is replayed.
In case of chaining of this operator, it has to be made sure that the operators in the chain
are opened tail to head. The reason for this is that an opened AsyncWaitOperator starts
already emitting recovered StreamElement to downstream operators.
userFunctionchainingStrategy, config, latencyStats, LOG, metrics, output, processingTimeService| Constructor and Description |
|---|
AsyncWaitOperator(AsyncFunction<IN,OUT> asyncFunction,
long timeout,
int capacity,
AsyncDataStream.OutputMode outputMode,
ProcessingTimeService processingTimeService,
MailboxExecutor mailboxExecutor) |
| Modifier and Type | Method and Description |
|---|---|
void |
endInput()
It is notified that no more data will arrive from the input.
|
void |
initializeState(org.apache.flink.runtime.state.StateInitializationContext context)
Stream operators with state which can be restored need to override this hook method.
|
void |
open()
This method is called immediately before any elements are processed, it should contain the
operator's initialization logic, e.g.
|
void |
processElement(StreamRecord<IN> record)
Processes one element that arrived on this input of the
MultipleInputStreamOperator. |
void |
processWatermark(Watermark mark)
Processes a
Watermark that arrived on the first input of this two-input operator. |
void |
setup(StreamTask<?,?> containingTask,
StreamConfig config,
Output<StreamRecord<OUT>> output)
Initializes the operator.
|
void |
snapshotState(org.apache.flink.runtime.state.StateSnapshotContext context)
Stream operators with state, which want to participate in a snapshot need to override this
hook method.
|
close, dispose, getUserFunction, getUserFunctionParameters, notifyCheckpointAborted, notifyCheckpointComplete, setOutputTypegetChainingStrategy, getContainingTask, getCurrentKey, getExecutionConfig, getInternalTimerService, getKeyedStateBackend, getKeyedStateStore, getMetricGroup, getOperatorConfig, getOperatorID, getOperatorName, getOperatorStateBackend, getOrCreateKeyedState, getPartitionedState, getPartitionedState, getProcessingTimeService, getRuntimeContext, getTimeServiceManager, getUserCodeClassloader, initializeState, isUsingCustomRawKeyedState, prepareSnapshotPreBarrier, processLatencyMarker, processLatencyMarker1, processLatencyMarker2, processWatermark1, processWatermark2, reportOrForwardLatencyMarker, setChainingStrategy, setCurrentKey, setKeyContextElement1, setKeyContextElement2, setProcessingTimeService, snapshotStateclone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, waitsetKeyContextElementclose, dispose, getMetricGroup, getOperatorID, initializeState, prepareSnapshotPreBarrier, setKeyContextElement1, setKeyContextElement2, snapshotStatenotifyCheckpointAborted, notifyCheckpointCompletegetCurrentKey, setCurrentKeyprocessLatencyMarkerpublic AsyncWaitOperator(@Nonnull AsyncFunction<IN,OUT> asyncFunction, long timeout, int capacity, @Nonnull AsyncDataStream.OutputMode outputMode, @Nonnull ProcessingTimeService processingTimeService, @Nonnull MailboxExecutor mailboxExecutor)
public void setup(StreamTask<?,?> containingTask, StreamConfig config, Output<StreamRecord<OUT>> output)
SetupableStreamOperatorsetup in interface SetupableStreamOperator<OUT>setup in class AbstractUdfStreamOperator<OUT,AsyncFunction<IN,OUT>>public void open()
throws Exception
AbstractStreamOperatorThe default implementation does nothing.
open in interface StreamOperator<OUT>open in class AbstractUdfStreamOperator<OUT,AsyncFunction<IN,OUT>>Exception - An exception in this method causes the operator to fail.public void processElement(StreamRecord<IN> record) throws Exception
InputMultipleInputStreamOperator.
This method is guaranteed to not be called concurrently with other methods of the operator.processElement in interface Input<IN>Exceptionpublic void processWatermark(Watermark mark) throws Exception
InputWatermark that arrived on the first input of this two-input operator.
This method is guaranteed to not be called concurrently with other methods of the operator.processWatermark in interface Input<IN>processWatermark in class AbstractStreamOperator<OUT>ExceptionWatermarkpublic void snapshotState(org.apache.flink.runtime.state.StateSnapshotContext context)
throws Exception
AbstractStreamOperatorsnapshotState in interface StreamOperatorStateHandler.CheckpointedStreamOperatorsnapshotState in class AbstractUdfStreamOperator<OUT,AsyncFunction<IN,OUT>>context - context that provides information and means required for taking a snapshotExceptionpublic void initializeState(org.apache.flink.runtime.state.StateInitializationContext context)
throws Exception
AbstractStreamOperatorinitializeState in interface StreamOperatorStateHandler.CheckpointedStreamOperatorinitializeState in class AbstractUdfStreamOperator<OUT,AsyncFunction<IN,OUT>>context - context that allows to register different states.Exceptionpublic void endInput()
throws Exception
BoundedOneInputWARNING: It is not safe to use this method to commit any transactions or other side
effects! You can use this method to flush any buffered data that can later on be committed
e.g. in a CheckpointListener.notifyCheckpointComplete(long).
endInput in interface BoundedOneInputExceptionCopyright © 2014–2022 The Apache Software Foundation. All rights reserved.