org.apache.flink.table.runtime.operators.window.slicing

类 SlicingWindowOperator<K,W>

java.lang.Object

org.apache.flink.streaming.api.operators.AbstractStreamOperator<OUT>

org.apache.flink.table.runtime.operators.TableStreamOperator<org.apache.flink.table.data.RowData>

org.apache.flink.table.runtime.operators.window.slicing.SlicingWindowOperator<K,W>

所有已实现的接口:

Serializable, org.apache.flink.api.common.state.CheckpointListener, org.apache.flink.streaming.api.operators.Input<org.apache.flink.table.data.RowData>, org.apache.flink.streaming.api.operators.KeyContext, org.apache.flink.streaming.api.operators.OneInputStreamOperator<org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData>, org.apache.flink.streaming.api.operators.SetupableStreamOperator<org.apache.flink.table.data.RowData>, org.apache.flink.streaming.api.operators.StreamOperator<org.apache.flink.table.data.RowData>, org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.CheckpointedStreamOperator, org.apache.flink.streaming.api.operators.Triggerable<K,W>

@Internal
public final class SlicingWindowOperator<K,W>
extends TableStreamOperator<org.apache.flink.table.data.RowData>
implements org.apache.flink.streaming.api.operators.OneInputStreamOperator<org.apache.flink.table.data.RowData,org.apache.flink.table.data.RowData>, org.apache.flink.streaming.api.operators.Triggerable<K,W>, org.apache.flink.streaming.api.operators.KeyContext

The SlicingWindowOperator implements an optimized processing for aligned windows which can apply the slicing optimization. The core idea of slicing optimization is to divide all elements from a data stream into a finite number of non-overlapping chunks (a.k.a. slices).

Concept of Aligned Window and Unaligned Window

We divide windows into 2 categories: Aligned Windows and Unaligned Windows.

Aligned Windows are windows have predetermined window boundaries and windows can be divided into finite number of non-overlapping chunks. The boundary of an aligned window is determined independently from the time characteristic of the data stream, or messages it receives. For example, hopping (sliding) window is an aligned window as the window boundaries are predetermined based on the window size and slide. Aligned windows include tumbling, hopping, cumulative windows.

Unaligned Windows are windows determined dynamically based on elements. For example, session window is an unaligned window as the window boundaries are determined based on the messages timestamps and their correlations. Currently, unaligned windows include session window only.

Because aligned windows can be divided into finite number of non-overlapping chunks (a.k.a. slices), which can apply efficient processing to share intermediate results.

Concept of Slice

Dividing a window of aligned windows into a finite number of non-overlapping chunks, where the chunks are slices. It has the following properties:

• An element must only belong to a single slice.
• Slices are non-overlapping, i.e. S_i and S_j should not have any shared elements if i != j.
• A window is consist of a finite number of slices.

Abstraction of Slicing Window Operator

A slicing window operator is a simple wrap of SlicingWindowProcessor. It delegates all the important methods to the underlying processor, where the processor can have different implementation for aggregate and topk or others.

A SlicingWindowProcessor usually leverages the SliceAssigner to assign slices and calculate based on the slices. See SliceSharedWindowAggProcessor as an example.

Note: since SlicingWindowProcessor leverages slicing optimization for aligned windows, therefore, it doesn't support unaligned windows, e.g. session window.

Note: currently, SlicingWindowOperator doesn't support early-fire and late-arrival. Thus late elements (elements belong to emitted windows) will be simply dropped.

另请参阅:

序列化表格

嵌套类概要

从类继承的嵌套类/接口 org.apache.flink.table.runtime.operators.TableStreamOperator

TableStreamOperator.ContextImpl

字段概要

字段

限定符和类型	字段和说明
protected org.apache.flink.streaming.api.operators.TimestampedCollector<org.apache.flink.table.data.RowData>	collector This is used for emitting elements with a given timestamp.

从类继承的字段 org.apache.flink.table.runtime.operators.TableStreamOperator

ctx, currentWatermark

从类继承的字段 org.apache.flink.streaming.api.operators.AbstractStreamOperator

chainingStrategy, config, latencyStats, LOG, metrics, output, processingTimeService

构造器概要

构造器

构造器和说明
SlicingWindowOperator(SlicingWindowProcessor<W> windowProcessor)

方法概要

限定符和类型	方法和说明
void	close()
org.apache.flink.metrics.Counter	getNumLateRecordsDropped()
org.apache.flink.metrics.Gauge<Long>	getWatermarkLatency()
void	onEventTime(org.apache.flink.streaming.api.operators.InternalTimer<K,W> timer)
void	onProcessingTime(org.apache.flink.streaming.api.operators.InternalTimer<K,W> timer)
void	open()
void	prepareSnapshotPreBarrier(long checkpointId)
void	processElement(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<org.apache.flink.table.data.RowData> element)
void	processWatermark(org.apache.flink.streaming.api.watermark.Watermark mark)

从类继承的方法 org.apache.flink.table.runtime.operators.TableStreamOperator

computeMemorySize

从类继承的方法 org.apache.flink.streaming.api.operators.AbstractStreamOperator

finish, getChainingStrategy, getContainingTask, getCurrentKey, getExecutionConfig, getInternalTimerService, getKeyedStateBackend, getKeyedStateStore, getMetricGroup, getOperatorConfig, getOperatorID, getOperatorName, getOperatorStateBackend, getOrCreateKeyedState, getPartitionedState, getPartitionedState, getProcessingTimeService, getRuntimeContext, getTimeServiceManager, getUserCodeClassloader, initializeState, initializeState, isUsingCustomRawKeyedState, notifyCheckpointAborted, notifyCheckpointComplete, processLatencyMarker, processLatencyMarker1, processLatencyMarker2, processWatermark1, processWatermark2, processWatermarkStatus, processWatermarkStatus1, processWatermarkStatus2, reportOrForwardLatencyMarker, setChainingStrategy, setCurrentKey, setKeyContextElement1, setKeyContextElement2, setProcessingTimeService, setup, snapshotState, snapshotState

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

从接口继承的方法 org.apache.flink.streaming.api.operators.OneInputStreamOperator

setKeyContextElement

从接口继承的方法 org.apache.flink.streaming.api.operators.StreamOperator

finish, getMetricGroup, getOperatorID, initializeState, setKeyContextElement1, setKeyContextElement2, snapshotState

从接口继承的方法 org.apache.flink.api.common.state.CheckpointListener

notifyCheckpointAborted, notifyCheckpointComplete

从接口继承的方法 org.apache.flink.streaming.api.operators.KeyContext

getCurrentKey, setCurrentKey

从接口继承的方法 org.apache.flink.streaming.api.operators.Input

processLatencyMarker, processWatermarkStatus

字段详细资料

collector

protected transient org.apache.flink.streaming.api.operators.TimestampedCollector<org.apache.flink.table.data.RowData> collector

This is used for emitting elements with a given timestamp.

构造器详细资料

SlicingWindowOperator

public SlicingWindowOperator(SlicingWindowProcessor<W> windowProcessor)

方法详细资料

open

public void open()
          throws Exception

指定者:

open 在接口中 org.apache.flink.streaming.api.operators.StreamOperator<org.apache.flink.table.data.RowData>

覆盖:

open 在类中 TableStreamOperator<org.apache.flink.table.data.RowData>

抛出:

Exception

close

public void close()
           throws Exception

指定者:

close 在接口中 org.apache.flink.streaming.api.operators.StreamOperator<org.apache.flink.table.data.RowData>

覆盖:

close 在类中 org.apache.flink.streaming.api.operators.AbstractStreamOperator<org.apache.flink.table.data.RowData>

抛出:

Exception

processElement

public void processElement(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<org.apache.flink.table.data.RowData> element)
                    throws Exception

指定者:

processElement 在接口中 org.apache.flink.streaming.api.operators.Input<org.apache.flink.table.data.RowData>

抛出:

Exception

processWatermark

public void processWatermark(org.apache.flink.streaming.api.watermark.Watermark mark)
                      throws Exception

指定者:

processWatermark 在接口中 org.apache.flink.streaming.api.operators.Input<org.apache.flink.table.data.RowData>

覆盖:

processWatermark 在类中 TableStreamOperator<org.apache.flink.table.data.RowData>

抛出:

Exception

onEventTime

public void onEventTime(org.apache.flink.streaming.api.operators.InternalTimer<K,W> timer)
                 throws Exception

指定者:

onEventTime 在接口中 org.apache.flink.streaming.api.operators.Triggerable<K,W>

抛出:

Exception

onProcessingTime

public void onProcessingTime(org.apache.flink.streaming.api.operators.InternalTimer<K,W> timer)
                      throws Exception

指定者:

onProcessingTime 在接口中 org.apache.flink.streaming.api.operators.Triggerable<K,W>

抛出:

Exception

prepareSnapshotPreBarrier

public void prepareSnapshotPreBarrier(long checkpointId)
                               throws Exception

指定者:

prepareSnapshotPreBarrier 在接口中 org.apache.flink.streaming.api.operators.StreamOperator<org.apache.flink.table.data.RowData>

覆盖:

prepareSnapshotPreBarrier 在类中 org.apache.flink.streaming.api.operators.AbstractStreamOperator<org.apache.flink.table.data.RowData>

抛出:

Exception

getNumLateRecordsDropped

@VisibleForTesting
public org.apache.flink.metrics.Counter getNumLateRecordsDropped()

getWatermarkLatency

@VisibleForTesting
public org.apache.flink.metrics.Gauge<Long> getWatermarkLatency()