org.apache.flink.streaming.api.runners.python.beam

类 BeamPythonFunctionRunner

java.lang.Object

org.apache.flink.streaming.api.runners.python.beam.BeamPythonFunctionRunner

所有已实现的接口:

PythonFunctionRunner

直接已知子类:

BeamDataStreamPythonFunctionRunner, BeamTableStatefulPythonFunctionRunner, BeamTableStatelessPythonFunctionRunner

@Internal
public abstract class BeamPythonFunctionRunner
extends Object
implements PythonFunctionRunner

A BeamPythonFunctionRunner used to execute Python functions.

字段概要

字段

限定符和类型	字段和说明
protected static org.slf4j.Logger	LOG
protected org.apache.beam.sdk.fn.data.FnDataReceiver<org.apache.beam.sdk.util.WindowedValue<byte[]>>	mainInputReceiver The receiver which forwards the input elements to a remote environment for processing.
protected LinkedBlockingQueue<byte[]>	resultBuffer The Python function execution result receiver.
protected org.apache.flink.api.java.tuple.Tuple2<byte[],Integer>	resultTuple The Python function execution result tuple.

构造器概要

构造器

构造器和说明

BeamPythonFunctionRunner(String taskName, PythonEnvironmentManager environmentManager, String functionUrn, Map<String,String> jobOptions, FlinkMetricContainer flinkMetricContainer, org.apache.flink.runtime.state.KeyedStateBackend keyedStateBackend, org.apache.flink.api.common.typeutils.TypeSerializer keySerializer, org.apache.flink.runtime.memory.MemoryManager memoryManager, double managedMemoryFraction)

方法概要

限定符和类型	方法和说明
void	close() Tear-down the Python function runner.
org.apache.beam.runners.fnexecution.control.JobBundleFactory	createJobBundleFactory(org.apache.beam.vendor.grpc.v1p26p0.com.google.protobuf.Struct pipelineOptions)
void	flush() Forces to finish the processing of the current bundle of elements.
protected abstract org.apache.beam.model.pipeline.v1.RunnerApi.Coder	getInputCoderProto()
protected abstract org.apache.beam.model.pipeline.v1.RunnerApi.Coder	getOutputCoderProto()
protected abstract byte[]	getUserDefinedFunctionsProtoBytes()
void	open(PythonConfig config) Prepares the Python function runner, such as preparing the Python execution environment, etc.
org.apache.flink.api.java.tuple.Tuple2<byte[],Integer>	pollResult() Retrieves the Python function result.
void	process(byte[] data) Executes the Python function with the input byte array.
protected void	startBundle()

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

字段详细资料

LOG

protected static final org.slf4j.Logger LOG

resultTuple

protected final org.apache.flink.api.java.tuple.Tuple2<byte[],Integer> resultTuple

The Python function execution result tuple.

resultBuffer

protected transient LinkedBlockingQueue<byte[]> resultBuffer

The Python function execution result receiver.

mainInputReceiver

protected transient org.apache.beam.sdk.fn.data.FnDataReceiver<org.apache.beam.sdk.util.WindowedValue<byte[]>> mainInputReceiver

The receiver which forwards the input elements to a remote environment for processing.

构造器详细资料

BeamPythonFunctionRunner

public BeamPythonFunctionRunner(String taskName,
                                PythonEnvironmentManager environmentManager,
                                String functionUrn,
                                Map<String,String> jobOptions,
                                FlinkMetricContainer flinkMetricContainer,
                                @Nullable
                                org.apache.flink.runtime.state.KeyedStateBackend keyedStateBackend,
                                @Nullable
                                org.apache.flink.api.common.typeutils.TypeSerializer keySerializer,
                                @Nullable
                                org.apache.flink.runtime.memory.MemoryManager memoryManager,
                                double managedMemoryFraction)

方法详细资料

open

public void open(PythonConfig config)
          throws Exception

从接口复制的说明: PythonFunctionRunner

Prepares the Python function runner, such as preparing the Python execution environment, etc.

指定者:

open 在接口中 PythonFunctionRunner

抛出:

Exception

close

public void close()
           throws Exception

从接口复制的说明: PythonFunctionRunner

Tear-down the Python function runner.

指定者:

close 在接口中 PythonFunctionRunner

抛出:

Exception

process

public void process(byte[] data)
             throws Exception

从接口复制的说明: PythonFunctionRunner

Executes the Python function with the input byte array.

指定者:

process 在接口中 PythonFunctionRunner

参数:

data - the byte array data.

抛出:

Exception

pollResult

public org.apache.flink.api.java.tuple.Tuple2<byte[],Integer> pollResult()
                                                                  throws Exception

从接口复制的说明: PythonFunctionRunner

Retrieves the Python function result.

指定者:

pollResult 在接口中 PythonFunctionRunner

返回:

the head of he Python function result buffer, or null if the result buffer is empty. f0 means the byte array buffer which stores the Python function result. f1 means the length of the Python function result byte array.

抛出:

Exception

flush

public void flush()
           throws Exception

从接口复制的说明: PythonFunctionRunner

Forces to finish the processing of the current bundle of elements. It will flush the data cached in the data buffer for processing and retrieves the state mutations (if exists) made by the Python function. The call blocks until all of the outputs produced by this bundle have been received.

指定者:

flush 在接口中 PythonFunctionRunner

抛出:

Exception

createJobBundleFactory

public org.apache.beam.runners.fnexecution.control.JobBundleFactory createJobBundleFactory(org.apache.beam.vendor.grpc.v1p26p0.com.google.protobuf.Struct pipelineOptions)
                                                                                    throws Exception

抛出:

Exception

startBundle

protected void startBundle()

getUserDefinedFunctionsProtoBytes

protected abstract byte[] getUserDefinedFunctionsProtoBytes()

getInputCoderProto

protected abstract org.apache.beam.model.pipeline.v1.RunnerApi.Coder getInputCoderProto()

getOutputCoderProto

protected abstract org.apache.beam.model.pipeline.v1.RunnerApi.Coder getOutputCoderProto()