org.apache.flink.table.runtime.operators.join.temporal

类 TemporalRowTimeJoinOperator

java.lang.Object

org.apache.flink.streaming.api.operators.AbstractStreamOperator<org.apache.flink.table.data.RowData>

org.apache.flink.table.runtime.operators.join.temporal.BaseTwoInputStreamOperatorWithStateRetention

org.apache.flink.table.runtime.operators.join.temporal.TemporalRowTimeJoinOperator

所有已实现的接口:

Serializable, org.apache.flink.api.common.state.CheckpointListener, org.apache.flink.streaming.api.operators.KeyContext, org.apache.flink.streaming.api.operators.SetupableStreamOperator<org.apache.flink.table.data.RowData>, org.apache.flink.streaming.api.operators.StreamOperator<org.apache.flink.table.data.RowData>, org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.CheckpointedStreamOperator, org.apache.flink.streaming.api.operators.Triggerable<Object,org.apache.flink.runtime.state.VoidNamespace>, org.apache.flink.streaming.api.operators.TwoInputStreamOperator<org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData>

public class TemporalRowTimeJoinOperator
extends BaseTwoInputStreamOperatorWithStateRetention

The operator for temporal join (FOR SYSTEM_TIME AS OF o.rowtime) on row time, it has no limitation about message types of the left input and right input, this means the operator deals changelog well.

For Event-time temporal join, its probe side is a regular table, its build side is a versioned table, the version of versioned table can extract from the build side state. This operator works by keeping on the state collection of probe and build records to process on next watermark. The idea is that between watermarks we are collecting those elements and once we are sure that there will be no updates we emit the correct result and clean up the expired data in state.

Cleaning up the state drops all of the "old" values from the probe side, where "old" is defined as older then the current watermark. Build side is also cleaned up in the similar fashion, however we always keep at least one record - the latest one - even if it's past the last watermark.

One more trick is how the emitting results and cleaning up is triggered. It is achieved by registering timers for the keys. We could register a timer for every probe and build side element's event time (when watermark exceeds this timer, that's when we are emitting and/or cleaning up the state). However this would cause huge number of registered timers. For example with following evenTimes of probe records accumulated: {1, 2, 5, 8, 9}, if we had received Watermark(10), it would trigger 5 separate timers for the same key. To avoid that we always keep only one single registered timer for any given key, registered for the minimal value. Upon triggering it, we process all records with event times older then or equal to currentWatermark.

另请参阅:

序列化表格

字段概要

从类继承的字段 org.apache.flink.table.runtime.operators.join.temporal.BaseTwoInputStreamOperatorWithStateRetention

stateCleaningEnabled

从类继承的字段 org.apache.flink.streaming.api.operators.AbstractStreamOperator

chainingStrategy, config, latencyStats, LOG, metrics, output, processingTimeService

构造器概要

构造器

构造器和说明
TemporalRowTimeJoinOperator(InternalTypeInfo<org.apache.flink.table.data.RowData> leftType, InternalTypeInfo<org.apache.flink.table.data.RowData> rightType, GeneratedJoinCondition generatedJoinCondition, int leftTimeAttribute, int rightTimeAttribute, long minRetentionTime, long maxRetentionTime, boolean isLeftOuterJoin)

方法概要

限定符和类型	方法和说明
void	cleanupState(long time) The method to be called when a cleanup timer fires.
void	close()
void	onEventTime(org.apache.flink.streaming.api.operators.InternalTimer<Object,org.apache.flink.runtime.state.VoidNamespace> timer)
void	open()
void	processElement1(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<org.apache.flink.table.data.RowData> element)
void	processElement2(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<org.apache.flink.table.data.RowData> element)

从类继承的方法 org.apache.flink.table.runtime.operators.join.temporal.BaseTwoInputStreamOperatorWithStateRetention

cleanupLastTimer, onProcessingTime, registerProcessingCleanupTimer

从类继承的方法 org.apache.flink.streaming.api.operators.AbstractStreamOperator

finish, getChainingStrategy, getContainingTask, getCurrentKey, getExecutionConfig, getInternalTimerService, getKeyedStateBackend, getKeyedStateStore, getMetricGroup, getOperatorConfig, getOperatorID, getOperatorName, getOperatorStateBackend, getOrCreateKeyedState, getPartitionedState, getPartitionedState, getProcessingTimeService, getRuntimeContext, getTimeServiceManager, getUserCodeClassloader, initializeState, initializeState, isUsingCustomRawKeyedState, notifyCheckpointAborted, notifyCheckpointComplete, prepareSnapshotPreBarrier, processLatencyMarker, processLatencyMarker1, processLatencyMarker2, processWatermark, processWatermark1, processWatermark2, processWatermarkStatus, processWatermarkStatus1, processWatermarkStatus2, reportOrForwardLatencyMarker, setChainingStrategy, setCurrentKey, setKeyContextElement1, setKeyContextElement2, setProcessingTimeService, setup, snapshotState, snapshotState

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

从接口继承的方法 org.apache.flink.streaming.api.operators.TwoInputStreamOperator

processLatencyMarker1, processLatencyMarker2, processWatermark1, processWatermark2, processWatermarkStatus1, processWatermarkStatus2

从接口继承的方法 org.apache.flink.streaming.api.operators.StreamOperator

finish, getMetricGroup, getOperatorID, initializeState, prepareSnapshotPreBarrier, setKeyContextElement1, setKeyContextElement2, snapshotState

从接口继承的方法 org.apache.flink.api.common.state.CheckpointListener

notifyCheckpointAborted, notifyCheckpointComplete

从接口继承的方法 org.apache.flink.streaming.api.operators.KeyContext

getCurrentKey, setCurrentKey

构造器详细资料

TemporalRowTimeJoinOperator

public TemporalRowTimeJoinOperator(InternalTypeInfo<org.apache.flink.table.data.RowData> leftType,
                                   InternalTypeInfo<org.apache.flink.table.data.RowData> rightType,
                                   GeneratedJoinCondition generatedJoinCondition,
                                   int leftTimeAttribute,
                                   int rightTimeAttribute,
                                   long minRetentionTime,
                                   long maxRetentionTime,
                                   boolean isLeftOuterJoin)

方法详细资料

open

public void open()
          throws Exception

指定者:

open 在接口中 org.apache.flink.streaming.api.operators.StreamOperator<org.apache.flink.table.data.RowData>

覆盖:

open 在类中 BaseTwoInputStreamOperatorWithStateRetention

抛出:

Exception

processElement1

public void processElement1(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<org.apache.flink.table.data.RowData> element)
                     throws Exception

抛出:

Exception

processElement2

public void processElement2(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<org.apache.flink.table.data.RowData> element)
                     throws Exception

抛出:

Exception

onEventTime

public void onEventTime(org.apache.flink.streaming.api.operators.InternalTimer<Object,org.apache.flink.runtime.state.VoidNamespace> timer)
                 throws Exception

抛出:

Exception

close

public void close()
           throws Exception

指定者:

close 在接口中 org.apache.flink.streaming.api.operators.StreamOperator<org.apache.flink.table.data.RowData>

覆盖:

close 在类中 org.apache.flink.streaming.api.operators.AbstractStreamOperator<org.apache.flink.table.data.RowData>

抛出:

Exception

cleanupState

public void cleanupState(long time)

The method to be called when a cleanup timer fires.

指定者:

cleanupState 在类中 BaseTwoInputStreamOperatorWithStateRetention

参数:

time - The timestamp of the fired timer.