org.apache.flink.table.runtime.operators.join.stream

Class MiniBatchStreamingJoinOperator

java.lang.Object

org.apache.flink.streaming.api.operators.AbstractStreamOperator<org.apache.flink.table.data.RowData>

org.apache.flink.table.runtime.operators.join.stream.AbstractStreamingJoinOperator

org.apache.flink.table.runtime.operators.join.stream.StreamingJoinOperator

org.apache.flink.table.runtime.operators.join.stream.MiniBatchStreamingJoinOperator

All Implemented Interfaces:

Serializable, org.apache.flink.api.common.state.CheckpointListener, org.apache.flink.streaming.api.operators.KeyContext, org.apache.flink.streaming.api.operators.KeyContextHandler, org.apache.flink.streaming.api.operators.SetupableStreamOperator<org.apache.flink.table.data.RowData>, org.apache.flink.streaming.api.operators.StreamOperator<org.apache.flink.table.data.RowData>, org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.CheckpointedStreamOperator, org.apache.flink.streaming.api.operators.TwoInputStreamOperator<org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData>, BundleTriggerCallback

public abstract class MiniBatchStreamingJoinOperator
extends StreamingJoinOperator
implements BundleTriggerCallback

Streaming unbounded Join base operator which support mini-batch join.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.flink.table.runtime.operators.join.stream.AbstractStreamingJoinOperator

AbstractStreamingJoinOperator.AssociatedRecords, AbstractStreamingJoinOperator.OuterRecord

Field Summary

Fields inherited from class org.apache.flink.table.runtime.operators.join.stream.StreamingJoinOperator

leftIsOuter, leftRecordStateView, rightIsOuter, rightRecordStateView

Fields inherited from class org.apache.flink.table.runtime.operators.join.stream.AbstractStreamingJoinOperator

collector, joinCondition, LEFT_RECORDS_STATE_NAME, leftInputSideSpec, leftStateRetentionTime, leftType, RIGHT_RECORDS_STATE_NAME, rightInputSideSpec, rightStateRetentionTime, rightType

Fields inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

chainingStrategy, config, lastRecordAttributes1, lastRecordAttributes2, latencyStats, LOG, metrics, output, processingTimeService

Constructor Summary

Constructors

Constructor and Description
MiniBatchStreamingJoinOperator(org.apache.flink.table.runtime.operators.join.stream.MiniBatchStreamingJoinOperator.MiniBatchStreamingJoinParameter parameter)

Method Summary

Modifier and Type	Method and Description
void	close()
void	finish()
void	finishBundle() This method is invoked to finish current bundle and start a new one when the trigger was fired.
static MiniBatchStreamingJoinOperator	newMiniBatchStreamJoinOperator(FlinkJoinType joinType, InternalTypeInfo<org.apache.flink.table.data.RowData> leftType, InternalTypeInfo<org.apache.flink.table.data.RowData> rightType, GeneratedJoinCondition generatedJoinCondition, JoinInputSideSpec leftInputSideSpec, JoinInputSideSpec rightInputSideSpec, boolean leftIsOuter, boolean rightIsOuter, boolean[] filterNullKeys, long leftStateRetentionTime, long rightStateRetentionTime, CoBundleTrigger<org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData> coBundleTrigger)
void	open()
void	prepareSnapshotPreBarrier(long checkpointId)
protected abstract void	processBundles(BufferBundle<?> leftBuffer, BufferBundle<?> rightBuffer)
void	processElement1(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<org.apache.flink.table.data.RowData> element)
void	processElement2(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<org.apache.flink.table.data.RowData> element)
protected void	processSingleSideBundles(BufferBundle<?> inputBuffer, JoinRecordStateView inputSideStateView, JoinRecordStateView otherSideStateView, boolean inputIsLeft) RetractMsg+accumulatingMsg would be optimized which would keep sending retractMsg but do not deal with state.
void	processWatermark1(org.apache.flink.streaming.api.watermark.Watermark mark)
void	processWatermark2(org.apache.flink.streaming.api.watermark.Watermark mark)

Methods inherited from class org.apache.flink.table.runtime.operators.join.stream.StreamingJoinOperator

processElement

Methods inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

getChainingStrategy, getContainingTask, getCurrentKey, getExecutionConfig, getInternalTimerService, getKeyedStateBackend, getKeyedStateStore, getMetricGroup, getOperatorConfig, getOperatorID, getOperatorName, getOperatorStateBackend, getOrCreateKeyedState, getPartitionedState, getPartitionedState, getProcessingTimeService, getRuntimeContext, getTimeServiceManager, getUserCodeClassloader, hasKeyContext1, hasKeyContext2, initializeState, initializeState, isUsingCustomRawKeyedState, notifyCheckpointAborted, notifyCheckpointComplete, processLatencyMarker, processLatencyMarker1, processLatencyMarker2, processRecordAttributes, processRecordAttributes1, processRecordAttributes2, processWatermark, processWatermarkStatus, processWatermarkStatus1, processWatermarkStatus2, reportOrForwardLatencyMarker, setChainingStrategy, setCurrentKey, setKeyContextElement1, setKeyContextElement2, setProcessingTimeService, setup, snapshotState, snapshotState

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.streaming.api.operators.TwoInputStreamOperator

processLatencyMarker1, processLatencyMarker2, processRecordAttributes1, processRecordAttributes2, processWatermarkStatus1, processWatermarkStatus2

Methods inherited from interface org.apache.flink.streaming.api.operators.StreamOperator

getMetricGroup, getOperatorID, initializeState, setKeyContextElement1, setKeyContextElement2, snapshotState

Methods inherited from interface org.apache.flink.api.common.state.CheckpointListener

notifyCheckpointAborted, notifyCheckpointComplete

Methods inherited from interface org.apache.flink.streaming.api.operators.KeyContext

getCurrentKey, setCurrentKey

Methods inherited from interface org.apache.flink.streaming.api.operators.KeyContextHandler

hasKeyContext

Constructor Detail

MiniBatchStreamingJoinOperator

public MiniBatchStreamingJoinOperator(org.apache.flink.table.runtime.operators.join.stream.MiniBatchStreamingJoinOperator.MiniBatchStreamingJoinParameter parameter)

Method Detail

open

public void open()
          throws Exception

Specified by:

open in interface org.apache.flink.streaming.api.operators.StreamOperator<org.apache.flink.table.data.RowData>

Overrides:

open in class StreamingJoinOperator

Throws:

Exception

processElement1

public void processElement1(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<org.apache.flink.table.data.RowData> element)
                     throws Exception

Specified by:

processElement1 in interface org.apache.flink.streaming.api.operators.TwoInputStreamOperator<org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData>

Overrides:

processElement1 in class StreamingJoinOperator

Throws:

Exception

processElement2

public void processElement2(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<org.apache.flink.table.data.RowData> element)
                     throws Exception

Specified by:

processElement2 in interface org.apache.flink.streaming.api.operators.TwoInputStreamOperator<org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData>

Overrides:

processElement2 in class StreamingJoinOperator

Throws:

Exception

processWatermark1

public void processWatermark1(org.apache.flink.streaming.api.watermark.Watermark mark)
                       throws Exception

Specified by:

processWatermark1 in interface org.apache.flink.streaming.api.operators.TwoInputStreamOperator<org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData>

Overrides:

processWatermark1 in class org.apache.flink.streaming.api.operators.AbstractStreamOperator<org.apache.flink.table.data.RowData>

Throws:

Exception

processWatermark2

public void processWatermark2(org.apache.flink.streaming.api.watermark.Watermark mark)
                       throws Exception

Specified by:

processWatermark2 in interface org.apache.flink.streaming.api.operators.TwoInputStreamOperator<org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData>

Overrides:

processWatermark2 in class org.apache.flink.streaming.api.operators.AbstractStreamOperator<org.apache.flink.table.data.RowData>

Throws:

Exception

prepareSnapshotPreBarrier

public void prepareSnapshotPreBarrier(long checkpointId)
                               throws Exception

Specified by:

prepareSnapshotPreBarrier in interface org.apache.flink.streaming.api.operators.StreamOperator<org.apache.flink.table.data.RowData>

Overrides:

prepareSnapshotPreBarrier in class org.apache.flink.streaming.api.operators.AbstractStreamOperator<org.apache.flink.table.data.RowData>

Throws:

Exception

finish

public void finish()
            throws Exception

Specified by:

finish in interface org.apache.flink.streaming.api.operators.StreamOperator<org.apache.flink.table.data.RowData>

Overrides:

finish in class org.apache.flink.streaming.api.operators.AbstractStreamOperator<org.apache.flink.table.data.RowData>

Throws:

Exception

close

public void close()
           throws Exception

Specified by:

close in interface org.apache.flink.streaming.api.operators.StreamOperator<org.apache.flink.table.data.RowData>

Overrides:

close in class AbstractStreamingJoinOperator

Throws:

Exception

finishBundle

public void finishBundle()
                  throws Exception

Description copied from interface: BundleTriggerCallback

This method is invoked to finish current bundle and start a new one when the trigger was fired.

Specified by:

finishBundle in interface BundleTriggerCallback

Throws:

Exception - This method may throw exceptions. Throwing an exception will cause the operation to fail and may trigger recovery.

processBundles

protected abstract void processBundles(BufferBundle<?> leftBuffer,
                                       BufferBundle<?> rightBuffer)
                                throws Exception

Throws:

Exception

processSingleSideBundles

protected void processSingleSideBundles(BufferBundle<?> inputBuffer,
                                        JoinRecordStateView inputSideStateView,
                                        JoinRecordStateView otherSideStateView,
                                        boolean inputIsLeft)
                                 throws Exception

RetractMsg+accumulatingMsg would be optimized which would keep sending retractMsg but do not deal with state.

Throws:

Exception

newMiniBatchStreamJoinOperator

public static MiniBatchStreamingJoinOperator newMiniBatchStreamJoinOperator(FlinkJoinType joinType,
                                                                            InternalTypeInfo<org.apache.flink.table.data.RowData> leftType,
                                                                            InternalTypeInfo<org.apache.flink.table.data.RowData> rightType,
                                                                            GeneratedJoinCondition generatedJoinCondition,
                                                                            JoinInputSideSpec leftInputSideSpec,
                                                                            JoinInputSideSpec rightInputSideSpec,
                                                                            boolean leftIsOuter,
                                                                            boolean rightIsOuter,
                                                                            boolean[] filterNullKeys,
                                                                            long leftStateRetentionTime,
                                                                            long rightStateRetentionTime,
                                                                            CoBundleTrigger<org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData> coBundleTrigger)