org.apache.flink.streaming.api.operators.python

类 OneInputPythonFunctionOperator<IN,OUT,UDFIN,UDFOUT>

java.lang.Object

org.apache.flink.streaming.api.operators.AbstractStreamOperator<OUT>

org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator<OUT>

org.apache.flink.streaming.api.operators.python.AbstractOneInputPythonFunctionOperator<IN,OUT>

org.apache.flink.streaming.api.operators.python.OneInputPythonFunctionOperator<IN,OUT,UDFIN,UDFOUT>

所有已实现的接口:

Serializable, org.apache.flink.api.common.state.CheckpointListener, org.apache.flink.streaming.api.operators.BoundedOneInput, org.apache.flink.streaming.api.operators.Input<IN>, org.apache.flink.streaming.api.operators.KeyContext, org.apache.flink.streaming.api.operators.OneInputStreamOperator<IN,OUT>, org.apache.flink.streaming.api.operators.SetupableStreamOperator<OUT>, org.apache.flink.streaming.api.operators.StreamOperator<OUT>, org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.CheckpointedStreamOperator, org.apache.flink.util.Disposable

直接已知子类:

PythonFlatMapOperator, PythonMapOperator, PythonPartitionCustomOperator, PythonProcessOperator, PythonReduceOperator, PythonTimestampsAndWatermarksOperator

@Internal
public abstract class OneInputPythonFunctionOperator<IN,OUT,UDFIN,UDFOUT>
extends AbstractOneInputPythonFunctionOperator<IN,OUT>

OneInputPythonFunctionOperator is responsible for launching beam runner which will start a python harness to execute user defined python function.

The operator will buffer the timestamp of input elements in a queue, and set into the produced output element.

另请参阅:

序列化表格

字段概要

字段

限定符和类型	字段和说明
protected org.apache.flink.core.memory.ByteArrayInputStreamWithPos	bais Reusable InputStream used to holding the execution results to be deserialized.
protected org.apache.flink.core.memory.DataInputViewStreamWrapper	baisWrapper InputStream Wrapper.
protected org.apache.flink.core.memory.ByteArrayOutputStreamWithPos	baos Reusable OutputStream used to holding the serialized input elements.
protected org.apache.flink.core.memory.DataOutputViewStreamWrapper	baosWrapper OutputStream Wrapper.
protected org.apache.flink.streaming.api.operators.TimestampedCollector<OUT>	collector
protected static String	DATA_STREAM_STATELESS_FUNCTION_URN
protected Map<String,String>	jobOptions The options used to configure the Python worker process.

从类继承的字段 org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator

elementCount, maxBundleSize, pythonFunctionRunner

从类继承的字段 org.apache.flink.streaming.api.operators.AbstractStreamOperator

chainingStrategy, latencyStats, LOG, metrics, output, processingTimeService

构造器概要

构造器

构造器和说明
OneInputPythonFunctionOperator(org.apache.flink.configuration.Configuration config, org.apache.flink.api.common.typeinfo.TypeInformation<UDFIN> runnerInputTypeInfo, org.apache.flink.api.common.typeinfo.TypeInformation<UDFOUT> runnerOutputTypeInfo, DataStreamPythonFunctionInfo pythonFunctionInfo)

方法概要

限定符和类型	方法和说明
PythonFunctionRunner	createPythonFunctionRunner() Creates the PythonFunctionRunner which is responsible for Python user-defined function execution.
abstract String	getCoderUrn()
String	getFunctionUrn()
Map<String,String>	getInternalParameters()
org.apache.flink.table.functions.python.PythonEnv	getPythonEnv() Returns the PythonEnv used to create PythonEnvironmentManager..
void	open()
void	processElement(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<IN> element)

从类继承的方法 org.apache.flink.streaming.api.operators.python.AbstractOneInputPythonFunctionOperator

endInput

从类继承的方法 org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator

checkInvokeFinishBundleByCount, close, createPythonEnvironmentManager, dispose, emitResult, emitResults, getConfig, getFlinkMetricContainer, getPythonConfig, invokeFinishBundle, isBundleFinished, prepareSnapshotPreBarrier, processWatermark, setPythonConfig

从类继承的方法 org.apache.flink.streaming.api.operators.AbstractStreamOperator

getChainingStrategy, getContainingTask, getCurrentKey, getExecutionConfig, getInternalTimerService, getKeyedStateBackend, getKeyedStateStore, getMetricGroup, getOperatorConfig, getOperatorID, getOperatorName, getOperatorStateBackend, getOrCreateKeyedState, getPartitionedState, getPartitionedState, getProcessingTimeService, getRuntimeContext, getTimeServiceManager, getUserCodeClassloader, initializeState, initializeState, isUsingCustomRawKeyedState, notifyCheckpointAborted, notifyCheckpointComplete, processLatencyMarker, processLatencyMarker1, processLatencyMarker2, processWatermark1, processWatermark2, reportOrForwardLatencyMarker, setChainingStrategy, setCurrentKey, setKeyContextElement1, setKeyContextElement2, setProcessingTimeService, setup, snapshotState, snapshotState

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

从接口继承的方法 org.apache.flink.streaming.api.operators.OneInputStreamOperator

setKeyContextElement

从接口继承的方法 org.apache.flink.streaming.api.operators.StreamOperator

close, dispose, getMetricGroup, getOperatorID, initializeState, prepareSnapshotPreBarrier, setKeyContextElement1, setKeyContextElement2, snapshotState

从接口继承的方法 org.apache.flink.api.common.state.CheckpointListener

notifyCheckpointAborted, notifyCheckpointComplete

从接口继承的方法 org.apache.flink.streaming.api.operators.KeyContext

getCurrentKey, setCurrentKey

从接口继承的方法 org.apache.flink.streaming.api.operators.Input

processLatencyMarker, processWatermark

字段详细资料

DATA_STREAM_STATELESS_FUNCTION_URN

protected static final String DATA_STREAM_STATELESS_FUNCTION_URN

另请参阅:

常量字段值

jobOptions

protected final Map<String,String> jobOptions

The options used to configure the Python worker process.

bais

protected transient org.apache.flink.core.memory.ByteArrayInputStreamWithPos bais

Reusable InputStream used to holding the execution results to be deserialized.

baisWrapper

protected transient org.apache.flink.core.memory.DataInputViewStreamWrapper baisWrapper

InputStream Wrapper.

baos

protected transient org.apache.flink.core.memory.ByteArrayOutputStreamWithPos baos

Reusable OutputStream used to holding the serialized input elements.

baosWrapper

protected transient org.apache.flink.core.memory.DataOutputViewStreamWrapper baosWrapper

OutputStream Wrapper.

collector

protected transient org.apache.flink.streaming.api.operators.TimestampedCollector<OUT> collector

构造器详细资料

OneInputPythonFunctionOperator

public OneInputPythonFunctionOperator(org.apache.flink.configuration.Configuration config,
                                      org.apache.flink.api.common.typeinfo.TypeInformation<UDFIN> runnerInputTypeInfo,
                                      org.apache.flink.api.common.typeinfo.TypeInformation<UDFOUT> runnerOutputTypeInfo,
                                      DataStreamPythonFunctionInfo pythonFunctionInfo)

方法详细资料

open

public void open()
          throws Exception

指定者:

open 在接口中 org.apache.flink.streaming.api.operators.StreamOperator<OUT>

覆盖:

open 在类中 AbstractPythonFunctionOperator<OUT>

抛出:

Exception

createPythonFunctionRunner

public PythonFunctionRunner createPythonFunctionRunner()
                                                throws Exception

从类复制的说明: AbstractPythonFunctionOperator

Creates the PythonFunctionRunner which is responsible for Python user-defined function execution.

指定者:

createPythonFunctionRunner 在类中 AbstractPythonFunctionOperator<OUT>

抛出:

Exception

getPythonEnv

public org.apache.flink.table.functions.python.PythonEnv getPythonEnv()

从类复制的说明: AbstractPythonFunctionOperator

Returns the PythonEnv used to create PythonEnvironmentManager..

指定者:

getPythonEnv 在类中 AbstractPythonFunctionOperator<OUT>

processElement

public void processElement(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<IN> element)
                    throws Exception

抛出:

Exception

getFunctionUrn

public String getFunctionUrn()

getInternalParameters

public Map<String,String> getInternalParameters()

getCoderUrn

public abstract String getCoderUrn()