org.apache.flink.streaming.api.operators.python.process

类 ExternalPythonKeyedProcessOperator<OUT>

java.lang.Object

org.apache.flink.streaming.api.operators.AbstractStreamOperator<OUT>

org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator<OUT>

org.apache.flink.streaming.api.operators.python.process.AbstractExternalPythonFunctionOperator<OUT>

org.apache.flink.streaming.api.operators.python.process.AbstractExternalDataStreamPythonFunctionOperator<OUT>

org.apache.flink.streaming.api.operators.python.process.AbstractExternalOneInputPythonFunctionOperator<org.apache.flink.types.Row,OUT>

org.apache.flink.streaming.api.operators.python.process.ExternalPythonKeyedProcessOperator<OUT>

所有已实现的接口:

Serializable, org.apache.flink.api.common.state.CheckpointListener, org.apache.flink.api.java.typeutils.ResultTypeQueryable<OUT>, org.apache.flink.streaming.api.operators.BoundedOneInput, org.apache.flink.streaming.api.operators.Input<org.apache.flink.types.Row>, org.apache.flink.streaming.api.operators.KeyContext, org.apache.flink.streaming.api.operators.KeyContextHandler, org.apache.flink.streaming.api.operators.OneInputStreamOperator<org.apache.flink.types.Row,OUT>, DataStreamPythonFunctionOperator<OUT>, org.apache.flink.streaming.api.operators.SetupableStreamOperator<OUT>, org.apache.flink.streaming.api.operators.StreamOperator<OUT>, org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.CheckpointedStreamOperator, org.apache.flink.streaming.api.operators.Triggerable<org.apache.flink.types.Row,Object>, org.apache.flink.streaming.api.operators.YieldingOperator<OUT>

@Internal
public class ExternalPythonKeyedProcessOperator<OUT>
extends AbstractExternalOneInputPythonFunctionOperator<org.apache.flink.types.Row,OUT>
implements org.apache.flink.streaming.api.operators.Triggerable<org.apache.flink.types.Row,Object>

ExternalPythonKeyedProcessOperator is responsible for launching beam runner which will start a python harness to execute user defined python function. It is also able to handle the timer and state request from the python stateful user defined function.

另请参阅:

序列化表格

字段概要

从类继承的字段 org.apache.flink.streaming.api.operators.python.process.AbstractExternalOneInputPythonFunctionOperator

baos, baosWrapper

从类继承的字段 org.apache.flink.streaming.api.operators.python.process.AbstractExternalPythonFunctionOperator

pythonFunctionRunner

从类继承的字段 org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator

bundleFinishedCallback, config, elementCount, lastFinishBundleTime, maxBundleSize, systemEnvEnabled

从类继承的字段 org.apache.flink.streaming.api.operators.AbstractStreamOperator

chainingStrategy, lastRecordAttributes1, lastRecordAttributes2, latencyStats, LOG, metrics, output, processingTimeService, stateHandler, stateKeySelector1, stateKeySelector2, timeServiceManager

构造器概要

构造器

构造器和说明
ExternalPythonKeyedProcessOperator(org.apache.flink.configuration.Configuration config, DataStreamPythonFunctionInfo pythonFunctionInfo, org.apache.flink.api.java.typeutils.RowTypeInfo inputTypeInfo, org.apache.flink.api.common.typeinfo.TypeInformation<OUT> outputTypeInfo)
ExternalPythonKeyedProcessOperator(org.apache.flink.configuration.Configuration config, DataStreamPythonFunctionInfo pythonFunctionInfo, org.apache.flink.api.java.typeutils.RowTypeInfo inputTypeInfo, org.apache.flink.api.common.typeinfo.TypeInformation<OUT> outputTypeInfo, org.apache.flink.api.common.typeutils.TypeSerializer namespaceSerializer)

方法概要

限定符和类型	方法和说明
<T> AbstractExternalDataStreamPythonFunctionOperator<T>	copy(DataStreamPythonFunctionInfo pythonFunctionInfo, org.apache.flink.api.common.typeinfo.TypeInformation<T> outputTypeInfo) Make a copy of the DataStreamPythonFunctionOperator with the given pythonFunctionInfo and outputTypeInfo.
PythonFunctionRunner	createPythonFunctionRunner() Creates the PythonFunctionRunner which is responsible for Python user-defined function execution.
Object	getCurrentKey()
void	onEventTime(org.apache.flink.streaming.api.operators.InternalTimer<org.apache.flink.types.Row,Object> timer)
void	onProcessingTime(org.apache.flink.streaming.api.operators.InternalTimer<org.apache.flink.types.Row,Object> timer)
void	open()
void	processElement(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<org.apache.flink.types.Row> element)
void	setCurrentKey(Object key) As the beam state gRPC service will access the KeyedStateBackend in parallel with this operator, we must override this method to prevent changing the current key of the KeyedStateBackend while the beam service is handling requests.

从类继承的方法 org.apache.flink.streaming.api.operators.python.process.AbstractExternalOneInputPythonFunctionOperator

createInputCoderInfoDescriptor, createOutputCoderInfoDescriptor, emitResult, endInput, getInputTypeInfo, processElement

从类继承的方法 org.apache.flink.streaming.api.operators.python.process.AbstractExternalDataStreamPythonFunctionOperator

addSideOutputTags, createSideOutputCoderDescriptors, getInternalParameters, getOutputTagById, getProducedType, getPythonEnv, getPythonFunctionInfo, getSideOutputTags, getSideOutputTypeSerializerById, setNumPartitions

从类继承的方法 org.apache.flink.streaming.api.operators.python.process.AbstractExternalPythonFunctionOperator

close, createPythonEnvironmentManager, emitResults, invokeFinishBundle

从类继承的方法 org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator

checkInvokeFinishBundleByCount, finish, getConfiguration, getFlinkMetricContainer, isBundleFinished, prepareSnapshotPreBarrier, processWatermark

从类继承的方法 org.apache.flink.streaming.api.operators.AbstractStreamOperator

getChainingStrategy, getContainingTask, getExecutionConfig, getInternalTimerService, getKeyedStateBackend, getKeyedStateStore, getMetricGroup, getOperatorConfig, getOperatorID, getOperatorName, getOperatorStateBackend, getOrCreateKeyedState, getPartitionedState, getPartitionedState, getProcessingTimeService, getRuntimeContext, getStateKeySelector1, getStateKeySelector2, getTimeServiceManager, getUserCodeClassloader, hasKeyContext1, hasKeyContext2, initializeState, initializeState, isUsingCustomRawKeyedState, notifyCheckpointAborted, notifyCheckpointComplete, processLatencyMarker, processLatencyMarker1, processLatencyMarker2, processRecordAttributes, processRecordAttributes1, processRecordAttributes2, processWatermark1, processWatermark2, processWatermarkStatus, processWatermarkStatus1, processWatermarkStatus2, reportOrForwardLatencyMarker, setChainingStrategy, setKeyContextElement1, setKeyContextElement2, setMailboxExecutor, setProcessingTimeService, setup, snapshotState, snapshotState, useSplittableTimers

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

从接口继承的方法 org.apache.flink.streaming.api.operators.OneInputStreamOperator

setKeyContextElement

从接口继承的方法 org.apache.flink.streaming.api.operators.StreamOperator

close, finish, getMetricGroup, getOperatorAttributes, getOperatorID, initializeState, prepareSnapshotPreBarrier, setKeyContextElement1, setKeyContextElement2, snapshotState

从接口继承的方法 org.apache.flink.api.common.state.CheckpointListener

notifyCheckpointAborted, notifyCheckpointComplete

从接口继承的方法 org.apache.flink.streaming.api.operators.Input

processLatencyMarker, processRecordAttributes, processWatermark, processWatermarkStatus

从接口继承的方法 org.apache.flink.streaming.api.operators.KeyContextHandler

hasKeyContext

构造器详细资料

ExternalPythonKeyedProcessOperator

public ExternalPythonKeyedProcessOperator(org.apache.flink.configuration.Configuration config,
                                          DataStreamPythonFunctionInfo pythonFunctionInfo,
                                          org.apache.flink.api.java.typeutils.RowTypeInfo inputTypeInfo,
                                          org.apache.flink.api.common.typeinfo.TypeInformation<OUT> outputTypeInfo)

ExternalPythonKeyedProcessOperator

public ExternalPythonKeyedProcessOperator(org.apache.flink.configuration.Configuration config,
                                          DataStreamPythonFunctionInfo pythonFunctionInfo,
                                          org.apache.flink.api.java.typeutils.RowTypeInfo inputTypeInfo,
                                          org.apache.flink.api.common.typeinfo.TypeInformation<OUT> outputTypeInfo,
                                          org.apache.flink.api.common.typeutils.TypeSerializer namespaceSerializer)

方法详细资料

open

public void open()
          throws Exception

指定者:

open 在接口中 org.apache.flink.streaming.api.operators.StreamOperator<OUT>

覆盖:

open 在类中 AbstractExternalOneInputPythonFunctionOperator<org.apache.flink.types.Row,OUT>

抛出:

Exception

onEventTime

public void onEventTime(org.apache.flink.streaming.api.operators.InternalTimer<org.apache.flink.types.Row,Object> timer)
                 throws Exception

指定者:

onEventTime 在接口中 org.apache.flink.streaming.api.operators.Triggerable<org.apache.flink.types.Row,Object>

抛出:

Exception

onProcessingTime

public void onProcessingTime(org.apache.flink.streaming.api.operators.InternalTimer<org.apache.flink.types.Row,Object> timer)
                      throws Exception

指定者:

onProcessingTime 在接口中 org.apache.flink.streaming.api.operators.Triggerable<org.apache.flink.types.Row,Object>

抛出:

Exception

createPythonFunctionRunner

public PythonFunctionRunner createPythonFunctionRunner()
                                                throws Exception

从类复制的说明: AbstractExternalPythonFunctionOperator

Creates the PythonFunctionRunner which is responsible for Python user-defined function execution.

指定者:

createPythonFunctionRunner 在类中 AbstractExternalPythonFunctionOperator<OUT>

抛出:

Exception

processElement

public void processElement(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<org.apache.flink.types.Row> element)
                    throws Exception

指定者:

processElement 在接口中 org.apache.flink.streaming.api.operators.Input<org.apache.flink.types.Row>

抛出:

Exception

setCurrentKey

public void setCurrentKey(Object key)

As the beam state gRPC service will access the KeyedStateBackend in parallel with this operator, we must override this method to prevent changing the current key of the KeyedStateBackend while the beam service is handling requests.

指定者:

setCurrentKey 在接口中 org.apache.flink.streaming.api.operators.KeyContext

覆盖:

setCurrentKey 在类中 AbstractPythonFunctionOperator<OUT>

getCurrentKey

public Object getCurrentKey()

指定者:

getCurrentKey 在接口中 org.apache.flink.streaming.api.operators.KeyContext

覆盖:

getCurrentKey 在类中 org.apache.flink.streaming.api.operators.AbstractStreamOperator<OUT>

copy

public <T> AbstractExternalDataStreamPythonFunctionOperator<T> copy(DataStreamPythonFunctionInfo pythonFunctionInfo,
                                                                    org.apache.flink.api.common.typeinfo.TypeInformation<T> outputTypeInfo)

从接口复制的说明: DataStreamPythonFunctionOperator

Make a copy of the DataStreamPythonFunctionOperator with the given pythonFunctionInfo and outputTypeInfo. This is used for chaining optimization which may need to update the underlying pythonFunctionInfo and outputTypeInfo with the other fields not changed.

指定者:

copy 在接口中 DataStreamPythonFunctionOperator<OUT>