org.apache.beam.sdk.util

Class IdentityWindowFn<T>

java.lang.Object

org.apache.beam.sdk.transforms.windowing.WindowFn<T,W>

org.apache.beam.sdk.transforms.windowing.NonMergingWindowFn<T,BoundedWindow>

org.apache.beam.sdk.util.IdentityWindowFn<T>

All Implemented Interfaces:

java.io.Serializable, HasDisplayData

@Internal
public class IdentityWindowFn<T>
extends NonMergingWindowFn<T,BoundedWindow>

A WindowFn that leaves all associations between elements and windows unchanged.

This WindowFn is applied when elements must be passed through a GroupByKey, but should maintain their existing Window assignments. Because windows may have been merged, the earlier WindowFn may not appropriately maintain the existing window assignments. For example, if the earlier WindowFn merges windows, after a GroupByKey the WindowingStrategy uses InvalidWindows, and no further GroupByKey can be applied without applying a new WindowFn. This WindowFn allows existing window assignments to be maintained across a single group by key, at which point the earlier WindowingStrategy should be restored.

This WindowFn is an internal implementation detail of sdk-provided utilities, and should not be used by Pipeline writers.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.beam.sdk.transforms.windowing.WindowFn

WindowFn.AssignContext, WindowFn.MergeContext

Constructor Summary

Constructors

Constructor and Description
IdentityWindowFn(Coder<? extends BoundedWindow> coder)

Method Summary

Modifier and Type	Method and Description
boolean	assignsToOneWindow() Returns true if this WindowFn always assigns an element to exactly one window.
java.util.Collection<BoundedWindow>	assignWindows(WindowFn.AssignContext c) Given a timestamp and element, returns the set of windows into which it should be placed.
WindowMappingFn<BoundedWindow>	getDefaultWindowMappingFn() Returns the default WindowMappingFn to use to map main input windows to side input windows.
org.joda.time.Instant	getOutputTime(org.joda.time.Instant inputTimestamp, BoundedWindow window) Returns the output timestamp to use for data depending on the given inputTimestamp in the specified window.
boolean	isCompatible(WindowFn<?,?> other) Returns whether this performs the same merging as the given WindowFn.
void	verifyCompatibility(WindowFn<?,?> other) Throw IncompatibleWindowException if this WindowFn does not perform the same merging as the given $WindowFn.
Coder<BoundedWindow>	windowCoder() Returns the Coder used for serializing the windows used by this windowFn.

Methods inherited from class org.apache.beam.sdk.transforms.windowing.NonMergingWindowFn

isNonMerging, mergeWindows

Methods inherited from class org.apache.beam.sdk.transforms.windowing.WindowFn

getWindowTypeDescriptor, populateDisplayData

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

IdentityWindowFn

public IdentityWindowFn(Coder<? extends BoundedWindow> coder)

Method Detail

assignWindows

public java.util.Collection<BoundedWindow> assignWindows(WindowFn.AssignContext c)
                                                  throws java.lang.Exception

Description copied from class: WindowFn

Given a timestamp and element, returns the set of windows into which it should be placed.

Specified by:

assignWindows in class WindowFn<T,BoundedWindow>

Throws:

java.lang.Exception

isCompatible

public boolean isCompatible(WindowFn<?,?> other)

Description copied from class: WindowFn

Returns whether this performs the same merging as the given WindowFn.

Specified by:

isCompatible in class WindowFn<T,BoundedWindow>

verifyCompatibility

public void verifyCompatibility(WindowFn<?,?> other)
                         throws IncompatibleWindowException

Description copied from class: WindowFn

Throw IncompatibleWindowException if this WindowFn does not perform the same merging as the given $WindowFn.

Overrides:

verifyCompatibility in class WindowFn<T,BoundedWindow>

Throws:

IncompatibleWindowException - if compared WindowFns are not compatible.

windowCoder

public Coder<BoundedWindow> windowCoder()

Description copied from class: WindowFn

Returns the Coder used for serializing the windows used by this windowFn.

Specified by:

windowCoder in class WindowFn<T,BoundedWindow>

getDefaultWindowMappingFn

public WindowMappingFn<BoundedWindow> getDefaultWindowMappingFn()

Description copied from class: WindowFn

Returns the default WindowMappingFn to use to map main input windows to side input windows. This should accept arbitrary main input windows, and produce a BoundedWindow that can be produced by this WindowFn.

Specified by:

getDefaultWindowMappingFn in class WindowFn<T,BoundedWindow>

getOutputTime

public org.joda.time.Instant getOutputTime(org.joda.time.Instant inputTimestamp,
                                           BoundedWindow window)

Description copied from class: WindowFn

Returns the output timestamp to use for data depending on the given inputTimestamp in the specified window.

The result of this method must be between inputTimestamp and window.maxTimestamp() (inclusive on both sides).

This function must be monotonic across input timestamps. Specifically, if A < B, then getOutputTime(A, window) <= getOutputTime(B, window).

For a WindowFn that doesn't produce overlapping windows, this can (and typically should) just return inputTimestamp. In the presence of overlapping windows, it is suggested that the result in later overlapping windows is past the end of earlier windows so that the later windows don't prevent the watermark from progressing past the end of the earlier window.

Overrides:

getOutputTime in class WindowFn<T,BoundedWindow>

assignsToOneWindow

public boolean assignsToOneWindow()

Description copied from class: WindowFn

Returns true if this WindowFn always assigns an element to exactly one window.

If this varies per-element, or cannot be determined, conservatively return false.

By default, returns false.

Overrides:

assignsToOneWindow in class WindowFn<T,BoundedWindow>