org.apache.flink.streaming.api.operators.sort

Class SortingDataInput<T,K>

java.lang.Object

org.apache.flink.streaming.api.operators.sort.SortingDataInput<T,K>

Type Parameters:

T - The type of the value in incoming StreamRecords.

K - The type of the key.

All Implemented Interfaces:

Closeable, AutoCloseable, org.apache.flink.runtime.io.AvailabilityProvider, PushingAsyncDataInput<T>, StreamTaskInput<T>

public final class SortingDataInput<T,K>
extends Object
implements StreamTaskInput<T>

A StreamTaskInput which sorts in the incoming records from a chained input. It postpones emitting the records until it receives InputStatus.END_OF_INPUT from the chained input. After it is done it emits a single record at a time from the sorter.

The sorter uses binary comparison of keys, which are extracted and serialized when received from the chained input. Moreover the timestamps of incoming records are used for secondary ordering. For the comparison it uses either FixedLengthByteKeyComparator if the length of the serialized key is constant, or VariableLengthByteKeyComparator otherwise.

Watermarks, stream statuses, nor latency markers are propagated downstream as they do not make sense with buffered records. The input emits the largest watermark seen after all records.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.flink.streaming.runtime.io.PushingAsyncDataInput

PushingAsyncDataInput.DataOutput<T>

Nested classes/interfaces inherited from interface org.apache.flink.runtime.io.AvailabilityProvider

org.apache.flink.runtime.io.AvailabilityProvider.AvailabilityHelper

Field Summary

Fields inherited from interface org.apache.flink.streaming.runtime.io.StreamTaskInput

UNSPECIFIED

Fields inherited from interface org.apache.flink.runtime.io.AvailabilityProvider

AVAILABLE

Constructor Summary

Constructors

Constructor and Description

SortingDataInput(StreamTaskInput<T> wrappedInput, org.apache.flink.api.common.typeutils.TypeSerializer<T> typeSerializer, org.apache.flink.api.common.typeutils.TypeSerializer<K> keySerializer, org.apache.flink.api.java.functions.KeySelector<T,K> keySelector, org.apache.flink.runtime.memory.MemoryManager memoryManager, org.apache.flink.runtime.io.disk.iomanager.IOManager ioManager, boolean objectReuse, double managedMemoryFraction, org.apache.flink.configuration.Configuration jobConfiguration, org.apache.flink.runtime.jobgraph.tasks.AbstractInvokable containingTask)

Method Summary

Modifier and Type	Method and Description
void	close()
org.apache.flink.core.io.InputStatus	emitNext(PushingAsyncDataInput.DataOutput<T> output) Pushes the next element to the output from current data input, and returns the input status to indicate whether there are more available data in current input.
CompletableFuture<?>	getAvailableFuture()
int	getInputIndex() Returns the input index of this input.
CompletableFuture<Void>	prepareSnapshot(org.apache.flink.runtime.checkpoint.channel.ChannelStateWriter channelStateWriter, long checkpointId) Prepares to spill the in-flight input buffers as checkpoint snapshot.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.runtime.io.AvailabilityProvider

and, isApproximatelyAvailable, isAvailable, or

Constructor Detail

SortingDataInput

public SortingDataInput(StreamTaskInput<T> wrappedInput,
                        org.apache.flink.api.common.typeutils.TypeSerializer<T> typeSerializer,
                        org.apache.flink.api.common.typeutils.TypeSerializer<K> keySerializer,
                        org.apache.flink.api.java.functions.KeySelector<T,K> keySelector,
                        org.apache.flink.runtime.memory.MemoryManager memoryManager,
                        org.apache.flink.runtime.io.disk.iomanager.IOManager ioManager,
                        boolean objectReuse,
                        double managedMemoryFraction,
                        org.apache.flink.configuration.Configuration jobConfiguration,
                        org.apache.flink.runtime.jobgraph.tasks.AbstractInvokable containingTask)

Method Detail

getInputIndex

public int getInputIndex()

Description copied from interface: StreamTaskInput

Returns the input index of this input.

Specified by:

getInputIndex in interface StreamTaskInput<T>

prepareSnapshot

public CompletableFuture<Void> prepareSnapshot(org.apache.flink.runtime.checkpoint.channel.ChannelStateWriter channelStateWriter,
                                               long checkpointId)
                                        throws org.apache.flink.runtime.checkpoint.CheckpointException

Description copied from interface: StreamTaskInput

Prepares to spill the in-flight input buffers as checkpoint snapshot.

Specified by:

prepareSnapshot in interface StreamTaskInput<T>

Throws:

org.apache.flink.runtime.checkpoint.CheckpointException

close

public void close()
           throws IOException

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Throws:

IOException

emitNext

public org.apache.flink.core.io.InputStatus emitNext(PushingAsyncDataInput.DataOutput<T> output)
                                              throws Exception

Description copied from interface: PushingAsyncDataInput

Pushes the next element to the output from current data input, and returns the input status to indicate whether there are more available data in current input.

This method should be non blocking.

Specified by:

emitNext in interface PushingAsyncDataInput<T>

Throws:

Exception

getAvailableFuture

public CompletableFuture<?> getAvailableFuture()

Specified by:

getAvailableFuture in interface org.apache.flink.runtime.io.AvailabilityProvider