org.apache.flink.table.runtime.operators.python.aggregate.arrow.batch

Class BatchArrowPythonOverWindowAggregateFunctionOperator

java.lang.Object

org.apache.flink.streaming.api.operators.AbstractStreamOperator<OUT>

org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator<OUT>

org.apache.flink.streaming.api.operators.python.AbstractOneInputPythonFunctionOperator<IN,OUT>

org.apache.flink.table.runtime.operators.python.AbstractStatelessFunctionOperator<org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData>

org.apache.flink.table.runtime.operators.python.aggregate.arrow.AbstractArrowPythonAggregateFunctionOperator

org.apache.flink.table.runtime.operators.python.aggregate.arrow.batch.BatchArrowPythonOverWindowAggregateFunctionOperator

All Implemented Interfaces:

Serializable, org.apache.flink.api.common.state.CheckpointListener, org.apache.flink.streaming.api.operators.BoundedOneInput, org.apache.flink.streaming.api.operators.Input<org.apache.flink.table.data.RowData>, org.apache.flink.streaming.api.operators.KeyContext, org.apache.flink.streaming.api.operators.OneInputStreamOperator<org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData>, org.apache.flink.streaming.api.operators.SetupableStreamOperator<org.apache.flink.table.data.RowData>, org.apache.flink.streaming.api.operators.StreamOperator<org.apache.flink.table.data.RowData>, org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.CheckpointedStreamOperator, org.apache.flink.util.Disposable

@Internal
public class BatchArrowPythonOverWindowAggregateFunctionOperator
extends AbstractArrowPythonAggregateFunctionOperator

The Batch Arrow Python AggregateFunction Operator for Over Window Aggregation.

See Also:

Serialized Form

Field Summary

Fields inherited from class org.apache.flink.table.runtime.operators.python.aggregate.arrow.AbstractArrowPythonAggregateFunctionOperator

arrowSerializer, currentBatchCount, groupingSet, pandasAggFunctions, reuseJoinedRow, rowDataWrapper

Fields inherited from class org.apache.flink.table.runtime.operators.python.AbstractStatelessFunctionOperator

bais, baisWrapper, baos, baosWrapper, forwardedInputQueue, inputType, outputType, userDefinedFunctionInputOffsets, userDefinedFunctionInputType, userDefinedFunctionOutputType

Fields inherited from class org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator

elementCount, maxBundleSize, pythonFunctionRunner

Fields inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

chainingStrategy, latencyStats, LOG, metrics, output, processingTimeService

Constructor Summary

Constructors

Constructor and Description

BatchArrowPythonOverWindowAggregateFunctionOperator(org.apache.flink.configuration.Configuration config, org.apache.flink.table.functions.python.PythonFunctionInfo[] pandasAggFunctions, org.apache.flink.table.types.logical.RowType inputType, org.apache.flink.table.types.logical.RowType outputType, long[] lowerBoundary, long[] upperBoundary, boolean[] isRangeWindows, int[] aggWindowIndex, int[] groupKey, int[] groupingSet, int[] udafInputOffsets, int inputTimeFieldIndex, boolean asc)

Method Summary

Modifier and Type	Method and Description
void	bufferInput(org.apache.flink.table.data.RowData input) Buffers the specified input, it will be used to construct the operator result together with the user-defined function execution result.
void	close()
void	emitResult(org.apache.flink.api.java.tuple.Tuple2<byte[],Integer> resultTuple) Sends the execution result to the downstream operator.
void	endInput()
String	getFunctionUrn()
String	getInputOutputCoderUrn()
FlinkFnApi.UserDefinedFunctions	getUserDefinedFunctionsProto() Gets the proto representation of the Python user-defined functions to be executed.
protected void	invokeCurrentBatch()
void	open()
void	processElementInternal(org.apache.flink.table.data.RowData value)

Methods inherited from class org.apache.flink.table.runtime.operators.python.aggregate.arrow.AbstractArrowPythonAggregateFunctionOperator

dispose, getFunctionInput, getPythonEnv, isBundleFinished, processElement

Methods inherited from class org.apache.flink.table.runtime.operators.python.AbstractStatelessFunctionOperator

createPythonFunctionRunner

Methods inherited from class org.apache.flink.streaming.api.operators.python.AbstractPythonFunctionOperator

checkInvokeFinishBundleByCount, createPythonEnvironmentManager, emitResults, getConfig, getFlinkMetricContainer, getPythonConfig, invokeFinishBundle, prepareSnapshotPreBarrier, processWatermark, setCurrentKey, setPythonConfig

Methods inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

getChainingStrategy, getContainingTask, getCurrentKey, getExecutionConfig, getInternalTimerService, getKeyedStateBackend, getKeyedStateStore, getMetricGroup, getOperatorConfig, getOperatorID, getOperatorName, getOperatorStateBackend, getOrCreateKeyedState, getPartitionedState, getPartitionedState, getProcessingTimeService, getRuntimeContext, getTimeServiceManager, getUserCodeClassloader, initializeState, initializeState, isUsingCustomRawKeyedState, notifyCheckpointAborted, notifyCheckpointComplete, processLatencyMarker, processLatencyMarker1, processLatencyMarker2, processWatermark1, processWatermark2, reportOrForwardLatencyMarker, setChainingStrategy, setKeyContextElement1, setKeyContextElement2, setProcessingTimeService, setup, snapshotState, snapshotState

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.streaming.api.operators.OneInputStreamOperator

setKeyContextElement

Methods inherited from interface org.apache.flink.streaming.api.operators.StreamOperator

getMetricGroup, getOperatorID, initializeState, prepareSnapshotPreBarrier, setKeyContextElement1, setKeyContextElement2, snapshotState

Methods inherited from interface org.apache.flink.api.common.state.CheckpointListener

notifyCheckpointAborted, notifyCheckpointComplete

Methods inherited from interface org.apache.flink.streaming.api.operators.KeyContext

getCurrentKey, setCurrentKey

Methods inherited from interface org.apache.flink.streaming.api.operators.Input

processLatencyMarker, processWatermark

Constructor Detail

BatchArrowPythonOverWindowAggregateFunctionOperator

public BatchArrowPythonOverWindowAggregateFunctionOperator(org.apache.flink.configuration.Configuration config,
                                                           org.apache.flink.table.functions.python.PythonFunctionInfo[] pandasAggFunctions,
                                                           org.apache.flink.table.types.logical.RowType inputType,
                                                           org.apache.flink.table.types.logical.RowType outputType,
                                                           long[] lowerBoundary,
                                                           long[] upperBoundary,
                                                           boolean[] isRangeWindows,
                                                           int[] aggWindowIndex,
                                                           int[] groupKey,
                                                           int[] groupingSet,
                                                           int[] udafInputOffsets,
                                                           int inputTimeFieldIndex,
                                                           boolean asc)

Method Detail

open

public void open()
          throws Exception

Specified by:

open in interface org.apache.flink.streaming.api.operators.StreamOperator<org.apache.flink.table.data.RowData>

Throws:

Exception

bufferInput

public void bufferInput(org.apache.flink.table.data.RowData input)
                 throws Exception

Description copied from class: AbstractStatelessFunctionOperator

Buffers the specified input, it will be used to construct the operator result together with the user-defined function execution result.

Specified by:

bufferInput in class AbstractStatelessFunctionOperator<org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData>

Throws:

Exception

invokeCurrentBatch

protected void invokeCurrentBatch()
                           throws Exception

Throws:

Exception

processElementInternal

public void processElementInternal(org.apache.flink.table.data.RowData value)

Specified by:

processElementInternal in class AbstractStatelessFunctionOperator<org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData>

emitResult

public void emitResult(org.apache.flink.api.java.tuple.Tuple2<byte[],Integer> resultTuple)
                throws Exception

Description copied from class: AbstractPythonFunctionOperator

Sends the execution result to the downstream operator.

Specified by:

emitResult in class AbstractPythonFunctionOperator<org.apache.flink.table.data.RowData>

Throws:

Exception

getUserDefinedFunctionsProto

public FlinkFnApi.UserDefinedFunctions getUserDefinedFunctionsProto()

Description copied from class: AbstractStatelessFunctionOperator

Gets the proto representation of the Python user-defined functions to be executed.

Overrides:

getUserDefinedFunctionsProto in class AbstractArrowPythonAggregateFunctionOperator

getFunctionUrn

public String getFunctionUrn()

Overrides:

getFunctionUrn in class AbstractArrowPythonAggregateFunctionOperator

getInputOutputCoderUrn

public String getInputOutputCoderUrn()

Overrides:

getInputOutputCoderUrn in class AbstractArrowPythonAggregateFunctionOperator

endInput

public void endInput()
              throws Exception

Specified by:

endInput in interface org.apache.flink.streaming.api.operators.BoundedOneInput

Overrides:

endInput in class AbstractOneInputPythonFunctionOperator<org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData>

Throws:

Exception

close

public void close()
           throws Exception

Specified by:

close in interface org.apache.flink.streaming.api.operators.StreamOperator<org.apache.flink.table.data.RowData>

Overrides:

close in class AbstractPythonFunctionOperator<org.apache.flink.table.data.RowData>

Throws:

Exception