org.apache.flink.streaming.runtime.tasks

Class OperatorChain<OUT,OP extends StreamOperator<OUT>>

java.lang.Object

org.apache.flink.streaming.runtime.tasks.OperatorChain<OUT,OP>

Type Parameters:

OUT - The type of elements accepted by the chain, i.e., the input type of the chain's main operator.

All Implemented Interfaces:

Closeable, AutoCloseable, BoundedMultiInput, StreamStatusMaintainer, StreamStatusProvider

@Internal
public class OperatorChain<OUT,OP extends StreamOperator<OUT>>
extends Object
implements StreamStatusMaintainer, BoundedMultiInput, Closeable

The OperatorChain contains all operators that are executed as one chain within a single StreamTask.

The main entry point to the chain is it's mainOperator. mainOperator is driving the execution of the StreamTask, by pulling the records from network inputs and/or source inputs and pushing produced records to the remaining chained operators.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	OperatorChain.ChainedSource Wrapper class to access the chained sources and their's outputs.

Constructor Summary

Constructors

Constructor and Description
OperatorChain(StreamTask<OUT,OP> containingTask, org.apache.flink.runtime.io.network.api.writer.RecordWriterDelegate<org.apache.flink.runtime.plugable.SerializationDelegate<StreamRecord<OUT>>> recordWriterDelegate)

Method Summary

Modifier and Type	Method and Description
void	broadcastEvent(org.apache.flink.runtime.event.AbstractEvent event)
void	broadcastEvent(org.apache.flink.runtime.event.AbstractEvent event, boolean isPriorityEvent)
void	close() This method releases all resources of the record writer output.
protected void	closeOperators(StreamTaskActionExecutor actionExecutor) Closes all operators in a chain effect way.
void	dispatchOperatorEvent(org.apache.flink.runtime.jobgraph.OperatorID operator, org.apache.flink.util.SerializedValue<org.apache.flink.runtime.operators.coordination.OperatorEvent> event)
void	endInput(int inputId) Ends the main operator input specified by inputId).
void	flushOutputs() This method should be called before finishing the record emission, to make sure any data that is still buffered will be sent.
Iterable<StreamOperatorWrapper<?,?>>	getAllOperators() Returns an Iterable which traverses all operators in forward topological order.
Iterable<StreamOperatorWrapper<?,?>>	getAllOperators(boolean reverse) Returns an Iterable which traverses all operators in forward or reverse topological order.
Output<StreamRecord<?>>	getChainedSourceOutput(StreamConfig.SourceInputConfig sourceInput)
List<Output<StreamRecord<?>>>	getChainedSourceOutputs()
OP	getMainOperator()
WatermarkGaugeExposingOutput<StreamRecord<OUT>>	getMainOperatorOutput()
int	getNumberOfOperators()
org.apache.flink.runtime.operators.coordination.OperatorEventDispatcher	getOperatorEventDispatcher()
StreamTaskSourceInput<?>	getSourceTaskInput(StreamConfig.SourceInputConfig sourceInput)
List<StreamTaskSourceInput<?>>	getSourceTaskInputs()
RecordWriterOutput<?>[]	getStreamOutputs()
StreamStatus	getStreamStatus() Returns the current stream status.
protected void	initializeStateAndOpenOperators(StreamTaskStateInitializer streamTaskStateInitializer) Initialize state and open all operators in the chain from tail to heads, contrary to StreamOperator.close() which happens heads to tail (see closeOperators(StreamTaskActionExecutor)).
void	prepareSnapshotPreBarrier(long checkpointId)
void	setIgnoreEndOfInput(boolean ignoreEndOfInput)
void	toggleStreamStatus(StreamStatus status) Toggles the current stream status.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

OperatorChain

public OperatorChain(StreamTask<OUT,OP> containingTask,
                     org.apache.flink.runtime.io.network.api.writer.RecordWriterDelegate<org.apache.flink.runtime.plugable.SerializationDelegate<StreamRecord<OUT>>> recordWriterDelegate)

Method Detail

getStreamStatus

public StreamStatus getStreamStatus()

Description copied from interface: StreamStatusProvider

Returns the current stream status.

Specified by:

getStreamStatus in interface StreamStatusProvider

Returns:

current stream status.

getOperatorEventDispatcher

public org.apache.flink.runtime.operators.coordination.OperatorEventDispatcher getOperatorEventDispatcher()

dispatchOperatorEvent

public void dispatchOperatorEvent(org.apache.flink.runtime.jobgraph.OperatorID operator,
                                  org.apache.flink.util.SerializedValue<org.apache.flink.runtime.operators.coordination.OperatorEvent> event)
                           throws org.apache.flink.util.FlinkException

Throws:

org.apache.flink.util.FlinkException

toggleStreamStatus

public void toggleStreamStatus(StreamStatus status)

Description copied from interface: StreamStatusMaintainer

Toggles the current stream status. This method should only have effect if the supplied stream status is different from the current status.

Specified by:

toggleStreamStatus in interface StreamStatusMaintainer

Parameters:

status - the new status to toggle to

broadcastEvent

public void broadcastEvent(org.apache.flink.runtime.event.AbstractEvent event)
                    throws IOException

Throws:

IOException

broadcastEvent

public void broadcastEvent(org.apache.flink.runtime.event.AbstractEvent event,
                           boolean isPriorityEvent)
                    throws IOException

Throws:

IOException

prepareSnapshotPreBarrier

public void prepareSnapshotPreBarrier(long checkpointId)
                               throws Exception

Throws:

Exception

endInput

public void endInput(int inputId)
              throws Exception

Ends the main operator input specified by inputId).

Specified by:

endInput in interface BoundedMultiInput

Parameters:

inputId - the input ID starts from 1 which indicates the first input.

Throws:

Exception

initializeStateAndOpenOperators

protected void initializeStateAndOpenOperators(StreamTaskStateInitializer streamTaskStateInitializer)
                                        throws Exception

Initialize state and open all operators in the chain from tail to heads, contrary to StreamOperator.close() which happens heads to tail (see closeOperators(StreamTaskActionExecutor)).

Throws:

Exception

closeOperators

protected void closeOperators(StreamTaskActionExecutor actionExecutor)
                       throws Exception

Closes all operators in a chain effect way. Closing happens from heads to tail operator in the chain, contrary to StreamOperator.open() which happens tail to heads (see initializeStateAndOpenOperators(StreamTaskStateInitializer)).

Throws:

Exception

getStreamOutputs

public RecordWriterOutput<?>[] getStreamOutputs()

getAllOperators

public Iterable<StreamOperatorWrapper<?,?>> getAllOperators()

Returns an Iterable which traverses all operators in forward topological order.

getAllOperators

public Iterable<StreamOperatorWrapper<?,?>> getAllOperators(boolean reverse)

Returns an Iterable which traverses all operators in forward or reverse topological order.

getNumberOfOperators

public int getNumberOfOperators()

getMainOperatorOutput

public WatermarkGaugeExposingOutput<StreamRecord<OUT>> getMainOperatorOutput()

getChainedSourceOutput

public Output<StreamRecord<?>> getChainedSourceOutput(StreamConfig.SourceInputConfig sourceInput)

getChainedSourceOutputs

public List<Output<StreamRecord<?>>> getChainedSourceOutputs()

getSourceTaskInput

public StreamTaskSourceInput<?> getSourceTaskInput(StreamConfig.SourceInputConfig sourceInput)

getSourceTaskInputs

public List<StreamTaskSourceInput<?>> getSourceTaskInputs()

flushOutputs

public void flushOutputs()
                  throws IOException

This method should be called before finishing the record emission, to make sure any data that is still buffered will be sent. It also ensures that all data sending related exceptions are recognized.

Throws:

IOException - Thrown, if the buffered data cannot be pushed into the output streams.

close

public void close()
           throws IOException

This method releases all resources of the record writer output. It stops the output flushing thread (if there is one) and releases all buffers currently held by the output serializers.

This method should never fail.

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Throws:

IOException

getMainOperator

@Nullable
public OP getMainOperator()

setIgnoreEndOfInput

public void setIgnoreEndOfInput(boolean ignoreEndOfInput)