org.apache.flink.streaming.api.operators.python

Class PythonProcessOperator<IN,OUT>

java.lang.Object

org.apache.flink.streaming.api.operators.AbstractStreamOperator<OUT>

org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator<OUT>

org.apache.flink.streaming.api.operators.python.AbstractOneInputPythonFunctionOperator<IN,OUT>

org.apache.flink.streaming.api.operators.python.OneInputPythonFunctionOperator<IN,OUT,org.apache.flink.types.Row,OUT>

org.apache.flink.streaming.api.operators.python.PythonProcessOperator<IN,OUT>

All Implemented Interfaces:

Serializable, org.apache.flink.api.common.state.CheckpointListener, org.apache.flink.streaming.api.operators.BoundedOneInput, org.apache.flink.streaming.api.operators.Input<IN>, org.apache.flink.streaming.api.operators.KeyContext, org.apache.flink.streaming.api.operators.OneInputStreamOperator<IN,OUT>, org.apache.flink.streaming.api.operators.SetupableStreamOperator<OUT>, org.apache.flink.streaming.api.operators.StreamOperator<OUT>, org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.CheckpointedStreamOperator, org.apache.flink.util.Disposable

@Internal
public class PythonProcessOperator<IN,OUT>
extends OneInputPythonFunctionOperator<IN,OUT,org.apache.flink.types.Row,OUT>

PythonProcessOperator is responsible for launching beam runner which will start a python harness to execute user defined python ProcessFunction.

See Also:

Serialized Form

Field Summary

Fields inherited from class org.apache.flink.streaming.api.operators.python.OneInputPythonFunctionOperator

bais, baisWrapper, baos, baosWrapper, collector, DATA_STREAM_STATELESS_FUNCTION_URN, jobOptions

Fields inherited from class org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator

elementCount, maxBundleSize, pythonFunctionRunner

Fields inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

chainingStrategy, latencyStats, LOG, metrics, output, processingTimeService

Constructor Summary

Constructors

Constructor and Description
PythonProcessOperator(org.apache.flink.configuration.Configuration config, org.apache.flink.api.common.typeinfo.TypeInformation<IN> inputTypeInfo, org.apache.flink.api.common.typeinfo.TypeInformation<OUT> outputTypeInfo, DataStreamPythonFunctionInfo pythonFunctionInfo)

Method Summary

Modifier and Type	Method and Description
void	emitResult(org.apache.flink.api.java.tuple.Tuple2<byte[],Integer> resultTuple) Sends the execution result to the downstream operator.
String	getCoderUrn()
String	getFunctionUrn()
void	open()
void	processElement(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<IN> element)
void	processWatermark(org.apache.flink.streaming.api.watermark.Watermark mark)

Methods inherited from class org.apache.flink.streaming.api.operators.python.OneInputPythonFunctionOperator

createPythonFunctionRunner, getInternalParameters, getPythonEnv

Methods inherited from class org.apache.flink.streaming.api.operators.python.AbstractOneInputPythonFunctionOperator

endInput

Methods inherited from class org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator

checkInvokeFinishBundleByCount, close, createPythonEnvironmentManager, dispose, emitResults, getConfig, getFlinkMetricContainer, getPythonConfig, invokeFinishBundle, isBundleFinished, prepareSnapshotPreBarrier, setCurrentKey, setPythonConfig

Methods inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

getChainingStrategy, getContainingTask, getCurrentKey, getExecutionConfig, getInternalTimerService, getKeyedStateBackend, getKeyedStateStore, getMetricGroup, getOperatorConfig, getOperatorID, getOperatorName, getOperatorStateBackend, getOrCreateKeyedState, getPartitionedState, getPartitionedState, getProcessingTimeService, getRuntimeContext, getTimeServiceManager, getUserCodeClassloader, initializeState, initializeState, isUsingCustomRawKeyedState, notifyCheckpointAborted, notifyCheckpointComplete, processLatencyMarker, processLatencyMarker1, processLatencyMarker2, processWatermark1, processWatermark2, reportOrForwardLatencyMarker, setChainingStrategy, setKeyContextElement1, setKeyContextElement2, setProcessingTimeService, setup, snapshotState, snapshotState

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.streaming.api.operators.OneInputStreamOperator

setKeyContextElement

Methods inherited from interface org.apache.flink.streaming.api.operators.StreamOperator

close, dispose, getMetricGroup, getOperatorID, initializeState, prepareSnapshotPreBarrier, setKeyContextElement1, setKeyContextElement2, snapshotState

Methods inherited from interface org.apache.flink.api.common.state.CheckpointListener

notifyCheckpointAborted, notifyCheckpointComplete

Methods inherited from interface org.apache.flink.streaming.api.operators.KeyContext

getCurrentKey, setCurrentKey

Methods inherited from interface org.apache.flink.streaming.api.operators.Input

processLatencyMarker

Constructor Detail

PythonProcessOperator

public PythonProcessOperator(org.apache.flink.configuration.Configuration config,
                             org.apache.flink.api.common.typeinfo.TypeInformation<IN> inputTypeInfo,
                             org.apache.flink.api.common.typeinfo.TypeInformation<OUT> outputTypeInfo,
                             DataStreamPythonFunctionInfo pythonFunctionInfo)

Method Detail

open

public void open()
          throws Exception

Specified by:

open in interface org.apache.flink.streaming.api.operators.StreamOperator<OUT>

Overrides:

open in class OneInputPythonFunctionOperator<IN,OUT,org.apache.flink.types.Row,OUT>

Throws:

Exception

emitResult

public void emitResult(org.apache.flink.api.java.tuple.Tuple2<byte[],Integer> resultTuple)
                throws Exception

Description copied from class: AbstractPythonFunctionOperator

Sends the execution result to the downstream operator.

Specified by:

emitResult in class AbstractPythonFunctionOperator<OUT>

Throws:

Exception

processElement

public void processElement(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<IN> element)
                    throws Exception

Specified by:

processElement in interface org.apache.flink.streaming.api.operators.Input<IN>

Overrides:

processElement in class OneInputPythonFunctionOperator<IN,OUT,org.apache.flink.types.Row,OUT>

Throws:

Exception

processWatermark

public void processWatermark(org.apache.flink.streaming.api.watermark.Watermark mark)
                      throws Exception

Specified by:

processWatermark in interface org.apache.flink.streaming.api.operators.Input<IN>

Overrides:

processWatermark in class AbstractPythonFunctionOperator<OUT>

Throws:

Exception

getFunctionUrn

public String getFunctionUrn()

Overrides:

getFunctionUrn in class OneInputPythonFunctionOperator<IN,OUT,org.apache.flink.types.Row,OUT>

getCoderUrn

public String getCoderUrn()

Specified by:

getCoderUrn in class OneInputPythonFunctionOperator<IN,OUT,org.apache.flink.types.Row,OUT>