org.apache.flink.table.runtime.functions.python

Class AbstractPythonStatelessFunctionFlatMap

java.lang.Object

org.apache.flink.api.common.functions.AbstractRichFunction

org.apache.flink.api.common.functions.RichFlatMapFunction<org.apache.flink.types.Row,org.apache.flink.types.Row>

org.apache.flink.table.runtime.functions.python.AbstractPythonStatelessFunctionFlatMap

All Implemented Interfaces:

Serializable, org.apache.flink.api.common.functions.FlatMapFunction<org.apache.flink.types.Row,org.apache.flink.types.Row>, org.apache.flink.api.common.functions.Function, org.apache.flink.api.common.functions.RichFunction, org.apache.flink.api.java.typeutils.ResultTypeQueryable<org.apache.flink.types.Row>

Direct Known Subclasses:

AbstractPythonScalarFunctionFlatMap, PythonTableFunctionFlatMap

@Internal
public abstract class AbstractPythonStatelessFunctionFlatMap
extends org.apache.flink.api.common.functions.RichFlatMapFunction<org.apache.flink.types.Row,org.apache.flink.types.Row>
implements org.apache.flink.api.java.typeutils.ResultTypeQueryable<org.apache.flink.types.Row>

Base Python stateless RichFlatMapFunction used to invoke Python stateless functions for the old planner.

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
protected org.apache.flink.core.memory.ByteArrayInputStreamWithPos	bais Reusable InputStream used to holding the execution results to be deserialized.
protected org.apache.flink.core.memory.DataInputViewStreamWrapper	baisWrapper InputStream Wrapper.
protected org.apache.flink.core.memory.ByteArrayOutputStreamWithPos	baos Reusable OutputStream used to holding the serialized input elements.
protected LinkedBlockingQueue<org.apache.flink.types.Row>	forwardedInputQueue The queue holding the input elements for which the execution results have not been received.
protected org.apache.flink.api.common.typeutils.TypeSerializer<org.apache.flink.types.Row>	forwardedInputSerializer The type serializer for the forwarded fields.
protected org.apache.flink.table.types.logical.RowType	inputType The input logical type.
protected Map<String,String>	jobOptions The options used to configure the Python worker process.
protected org.apache.flink.table.types.logical.RowType	outputType The output logical type.
protected PythonFunctionRunner	pythonFunctionRunner The PythonFunctionRunner which is responsible for Python user-defined function execution.
protected org.apache.flink.util.Collector<org.apache.flink.types.Row>	resultCollector The collector used to collect records.
protected org.apache.flink.table.types.logical.RowType	userDefinedFunctionInputType The user-defined function input logical type.
protected org.apache.flink.table.types.logical.RowType	userDefinedFunctionOutputType The user-defined function output logical type.

Constructor Summary

Constructors

Constructor and Description
AbstractPythonStatelessFunctionFlatMap(org.apache.flink.configuration.Configuration config, org.apache.flink.table.types.logical.RowType inputType, org.apache.flink.table.types.logical.RowType outputType, int[] userDefinedFunctionInputOffsets)

Method Summary

Modifier and Type	Method and Description
abstract void	bufferInput(org.apache.flink.types.Row input)
protected void	checkInvokeFinishBundleByCount() Checks whether to invoke finishBundle by elements count.
void	close()
protected PythonEnvironmentManager	createPythonEnvironmentManager()
abstract void	emitResult(org.apache.flink.api.java.tuple.Tuple2<byte[],Integer> resultTuple)
void	flatMap(org.apache.flink.types.Row value, org.apache.flink.util.Collector<org.apache.flink.types.Row> out)
protected FlinkMetricContainer	getFlinkMetricContainer()
abstract int	getForwardedFieldsCount()
protected org.apache.flink.types.Row	getFunctionInput(org.apache.flink.types.Row element)
abstract String	getFunctionUrn()
abstract String	getInputOutputCoderUrn()
org.apache.flink.api.common.typeinfo.TypeInformation<org.apache.flink.types.Row>	getProducedType()
protected PythonConfig	getPythonConfig()
abstract org.apache.flink.table.functions.python.PythonEnv	getPythonEnv() Returns the PythonEnv used to create PythonEnvironmentManager..
abstract FlinkFnApi.UserDefinedFunctions	getUserDefinedFunctionsProto() Gets the proto representation of the Python user-defined functions to be executed.
protected void	invokeFinishBundle()
void	open(org.apache.flink.configuration.Configuration parameters)
abstract void	processElementInternal(org.apache.flink.types.Row value)

Methods inherited from class org.apache.flink.api.common.functions.AbstractRichFunction

getIterationRuntimeContext, getRuntimeContext, setRuntimeContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

inputType

protected final org.apache.flink.table.types.logical.RowType inputType

The input logical type.

outputType

protected final org.apache.flink.table.types.logical.RowType outputType

The output logical type.

jobOptions

protected final Map<String,String> jobOptions

The options used to configure the Python worker process.

userDefinedFunctionInputType

protected transient org.apache.flink.table.types.logical.RowType userDefinedFunctionInputType

The user-defined function input logical type.

userDefinedFunctionOutputType

protected transient org.apache.flink.table.types.logical.RowType userDefinedFunctionOutputType

The user-defined function output logical type.

forwardedInputQueue

protected transient LinkedBlockingQueue<org.apache.flink.types.Row> forwardedInputQueue

The queue holding the input elements for which the execution results have not been received.

resultCollector

protected transient org.apache.flink.util.Collector<org.apache.flink.types.Row> resultCollector

The collector used to collect records.

pythonFunctionRunner

protected transient PythonFunctionRunner pythonFunctionRunner

The PythonFunctionRunner which is responsible for Python user-defined function execution.

bais

protected transient org.apache.flink.core.memory.ByteArrayInputStreamWithPos bais

Reusable InputStream used to holding the execution results to be deserialized.

baisWrapper

protected transient org.apache.flink.core.memory.DataInputViewStreamWrapper baisWrapper

InputStream Wrapper.

forwardedInputSerializer

protected transient org.apache.flink.api.common.typeutils.TypeSerializer<org.apache.flink.types.Row> forwardedInputSerializer

The type serializer for the forwarded fields.

baos

protected transient org.apache.flink.core.memory.ByteArrayOutputStreamWithPos baos

Reusable OutputStream used to holding the serialized input elements.

Constructor Detail

AbstractPythonStatelessFunctionFlatMap

public AbstractPythonStatelessFunctionFlatMap(org.apache.flink.configuration.Configuration config,
                                              org.apache.flink.table.types.logical.RowType inputType,
                                              org.apache.flink.table.types.logical.RowType outputType,
                                              int[] userDefinedFunctionInputOffsets)

Method Detail

getPythonConfig

protected PythonConfig getPythonConfig()

open

public void open(org.apache.flink.configuration.Configuration parameters)
          throws Exception

Specified by:

open in interface org.apache.flink.api.common.functions.RichFunction

Overrides:

open in class org.apache.flink.api.common.functions.AbstractRichFunction

Throws:

Exception

flatMap

public void flatMap(org.apache.flink.types.Row value,
                    org.apache.flink.util.Collector<org.apache.flink.types.Row> out)
             throws Exception

Specified by:

flatMap in interface org.apache.flink.api.common.functions.FlatMapFunction<org.apache.flink.types.Row,org.apache.flink.types.Row>

Specified by:

flatMap in class org.apache.flink.api.common.functions.RichFlatMapFunction<org.apache.flink.types.Row,org.apache.flink.types.Row>

Throws:

Exception

getProducedType

public org.apache.flink.api.common.typeinfo.TypeInformation<org.apache.flink.types.Row> getProducedType()

Specified by:

getProducedType in interface org.apache.flink.api.java.typeutils.ResultTypeQueryable<org.apache.flink.types.Row>

close

public void close()
           throws Exception

Specified by:

close in interface org.apache.flink.api.common.functions.RichFunction

Overrides:

close in class org.apache.flink.api.common.functions.AbstractRichFunction

Throws:

Exception

getPythonEnv

public abstract org.apache.flink.table.functions.python.PythonEnv getPythonEnv()

Returns the PythonEnv used to create PythonEnvironmentManager..

bufferInput

public abstract void bufferInput(org.apache.flink.types.Row input)

emitResult

public abstract void emitResult(org.apache.flink.api.java.tuple.Tuple2<byte[],Integer> resultTuple)
                         throws Exception

Throws:

Exception

getForwardedFieldsCount

public abstract int getForwardedFieldsCount()

getUserDefinedFunctionsProto

public abstract FlinkFnApi.UserDefinedFunctions getUserDefinedFunctionsProto()

Gets the proto representation of the Python user-defined functions to be executed.

getInputOutputCoderUrn

public abstract String getInputOutputCoderUrn()

getFunctionUrn

public abstract String getFunctionUrn()

processElementInternal

public abstract void processElementInternal(org.apache.flink.types.Row value)
                                     throws Exception

Throws:

Exception

checkInvokeFinishBundleByCount

protected void checkInvokeFinishBundleByCount()
                                       throws Exception

Checks whether to invoke finishBundle by elements count. Called in flatMap.

Throws:

Exception

createPythonEnvironmentManager

protected PythonEnvironmentManager createPythonEnvironmentManager()
                                                           throws IOException

Throws:

IOException

getFlinkMetricContainer

protected FlinkMetricContainer getFlinkMetricContainer()

getFunctionInput

protected org.apache.flink.types.Row getFunctionInput(org.apache.flink.types.Row element)

invokeFinishBundle

protected void invokeFinishBundle()
                           throws Exception

Throws:

Exception