org.apache.flink.runtime.operators.sort

Class CombiningUnilateralSortMerger.CombiningSpillingThread

java.lang.Object

java.lang.Thread

org.apache.flink.runtime.operators.sort.UnilateralSortMerger.ThreadBase<E>

org.apache.flink.runtime.operators.sort.UnilateralSortMerger.SpillingThread

org.apache.flink.runtime.operators.sort.CombiningUnilateralSortMerger.CombiningSpillingThread

All Implemented Interfaces:

Runnable, Thread.UncaughtExceptionHandler

Enclosing class:

CombiningUnilateralSortMerger<E>

protected class CombiningUnilateralSortMerger.CombiningSpillingThread
extends UnilateralSortMerger.SpillingThread

Nested Class Summary

Nested classes/interfaces inherited from class java.lang.Thread

Thread.State, Thread.UncaughtExceptionHandler

Field Summary

Fields inherited from class org.apache.flink.runtime.operators.sort.UnilateralSortMerger.SpillingThread

comparator, ioManager, maxFanIn, memManager, mergeReadMemory, numWriteBuffersToCluster, serializer, writeMemory

Fields inherited from class org.apache.flink.runtime.operators.sort.UnilateralSortMerger.ThreadBase

queues

Fields inherited from class java.lang.Thread

MAX_PRIORITY, MIN_PRIORITY, NORM_PRIORITY

Constructor Summary

Constructors

Constructor and Description

CombiningSpillingThread(ExceptionHandler<IOException> exceptionHandler, UnilateralSortMerger.CircularQueues<E> queues, AbstractInvokable parentTask, MemoryManager memManager, IOManager ioManager, org.apache.flink.api.common.typeutils.TypeSerializer<E> serializer, org.apache.flink.api.common.typeutils.TypeComparator<E> comparator, List<org.apache.flink.core.memory.MemorySegment> sortReadMemory, List<org.apache.flink.core.memory.MemorySegment> writeMemory, int maxNumFileHandles, boolean objectReuseEnabled)

Method Summary

Modifier and Type	Method and Description
void	go() Entry point of the thread.
protected UnilateralSortMerger.ChannelWithBlockCount	mergeChannels(List<UnilateralSortMerger.ChannelWithBlockCount> channelIDs, List<List<org.apache.flink.core.memory.MemorySegment>> readBuffers, List<org.apache.flink.core.memory.MemorySegment> writeBuffers) Merges the sorted runs described by the given Channel IDs into a single sorted run.

Methods inherited from class org.apache.flink.runtime.operators.sort.UnilateralSortMerger.SpillingThread

disposeSortBuffers, getMergingIterator, getSegmentsForReaders, mergeChannelList, registerChannelToBeRemovedAtShudown, registerOpenChannelToBeRemovedAtShudown, takeNext, unregisterChannelToBeRemovedAtShudown, unregisterOpenChannelToBeRemovedAtShudown

Methods inherited from class org.apache.flink.runtime.operators.sort.UnilateralSortMerger.ThreadBase

internalHandleException, isRunning, run, shutdown, uncaughtException

Methods inherited from class java.lang.Thread

activeCount, checkAccess, clone, countStackFrames, currentThread, destroy, dumpStack, enumerate, getAllStackTraces, getContextClassLoader, getDefaultUncaughtExceptionHandler, getId, getName, getPriority, getStackTrace, getState, getThreadGroup, getUncaughtExceptionHandler, holdsLock, interrupt, interrupted, isAlive, isDaemon, isInterrupted, join, join, join, resume, setContextClassLoader, setDaemon, setDefaultUncaughtExceptionHandler, setName, setPriority, setUncaughtExceptionHandler, sleep, sleep, start, stop, stop, suspend, toString, yield

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

CombiningSpillingThread

public CombiningSpillingThread(ExceptionHandler<IOException> exceptionHandler,
                               UnilateralSortMerger.CircularQueues<E> queues,
                               AbstractInvokable parentTask,
                               MemoryManager memManager,
                               IOManager ioManager,
                               org.apache.flink.api.common.typeutils.TypeSerializer<E> serializer,
                               org.apache.flink.api.common.typeutils.TypeComparator<E> comparator,
                               List<org.apache.flink.core.memory.MemorySegment> sortReadMemory,
                               List<org.apache.flink.core.memory.MemorySegment> writeMemory,
                               int maxNumFileHandles,
                               boolean objectReuseEnabled)

Method Detail

go

public void go()
        throws IOException

Entry point of the thread.

Overrides:

go in class UnilateralSortMerger.SpillingThread

Throws:

IOException - Exceptions that prohibit correct completion of the work may be thrown by the thread.

mergeChannels

protected UnilateralSortMerger.ChannelWithBlockCount mergeChannels(List<UnilateralSortMerger.ChannelWithBlockCount> channelIDs,
                                                                   List<List<org.apache.flink.core.memory.MemorySegment>> readBuffers,
                                                                   List<org.apache.flink.core.memory.MemorySegment> writeBuffers)
                                                            throws IOException

Merges the sorted runs described by the given Channel IDs into a single sorted run. The merging process uses the given read and write buffers. During the merging process, the combiner is used to reduce the number of values with identical key.

Overrides:

mergeChannels in class UnilateralSortMerger.SpillingThread

Parameters:

channelIDs - The IDs of the runs' channels.

readBuffers - The buffers for the readers that read the sorted runs.

writeBuffers - The buffers for the writer that writes the merged channel.

Returns:

The ID of the channel that describes the merged run.

Throws:

IOException