org.apache.flink.table.runtime.operators.sort

类 BinaryExternalSorter

java.lang.Object

org.apache.flink.table.runtime.operators.sort.BinaryExternalSorter

所有已实现的接口:

Closeable, AutoCloseable, org.apache.flink.runtime.operators.sort.Sorter<org.apache.flink.table.data.binary.BinaryRowData>, org.apache.flink.runtime.operators.util.CloseableInputProvider<org.apache.flink.table.data.binary.BinaryRowData>

public class BinaryExternalSorter
extends Object
implements org.apache.flink.runtime.operators.sort.Sorter<org.apache.flink.table.data.binary.BinaryRowData>

The BinaryExternalSorter is a full fledged sorter for binary format. It implements a multi-way merge sort. Internally, it has three asynchronous threads (sort, spill, merger) which communicate through a set of blocking circularQueues, forming a closed loop. Memory is allocated using the MemoryManager interface. Thus the component will not exceed the provided memory limits.

构造器概要

构造器

构造器和说明
BinaryExternalSorter(Object owner, org.apache.flink.runtime.memory.MemoryManager memoryManager, long reservedMemorySize, org.apache.flink.runtime.io.disk.iomanager.IOManager ioManager, AbstractRowDataSerializer<org.apache.flink.table.data.RowData> inputSerializer, BinaryRowDataSerializer serializer, NormalizedKeyComputer normalizedKeyComputer, RecordComparator comparator, org.apache.flink.configuration.Configuration conf)
BinaryExternalSorter(Object owner, org.apache.flink.runtime.memory.MemoryManager memoryManager, long reservedMemorySize, org.apache.flink.runtime.io.disk.iomanager.IOManager ioManager, AbstractRowDataSerializer<org.apache.flink.table.data.RowData> inputSerializer, BinaryRowDataSerializer serializer, NormalizedKeyComputer normalizedKeyComputer, RecordComparator comparator, org.apache.flink.configuration.Configuration conf, float startSpillingFraction)

方法概要

限定符和类型	方法和说明
void	close() Shuts down all the threads initiated by this sorter.
org.apache.flink.util.MutableObjectIterator<org.apache.flink.table.data.binary.BinaryRowData>	getIterator()
long	getNumSpillFiles()
long	getSpillInBytes()
long	getUsedMemoryInBytes()
void	startThreads() Starts all the threads that are used by this sorter.
void	write(org.apache.flink.util.MutableObjectIterator<org.apache.flink.table.data.binary.BinaryRowData> iterator)
void	write(org.apache.flink.table.data.RowData current)

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

构造器详细资料

BinaryExternalSorter

public BinaryExternalSorter(Object owner,
                            org.apache.flink.runtime.memory.MemoryManager memoryManager,
                            long reservedMemorySize,
                            org.apache.flink.runtime.io.disk.iomanager.IOManager ioManager,
                            AbstractRowDataSerializer<org.apache.flink.table.data.RowData> inputSerializer,
                            BinaryRowDataSerializer serializer,
                            NormalizedKeyComputer normalizedKeyComputer,
                            RecordComparator comparator,
                            org.apache.flink.configuration.Configuration conf)

BinaryExternalSorter

public BinaryExternalSorter(Object owner,
                            org.apache.flink.runtime.memory.MemoryManager memoryManager,
                            long reservedMemorySize,
                            org.apache.flink.runtime.io.disk.iomanager.IOManager ioManager,
                            AbstractRowDataSerializer<org.apache.flink.table.data.RowData> inputSerializer,
                            BinaryRowDataSerializer serializer,
                            NormalizedKeyComputer normalizedKeyComputer,
                            RecordComparator comparator,
                            org.apache.flink.configuration.Configuration conf,
                            float startSpillingFraction)

方法详细资料

startThreads

public void startThreads()

Starts all the threads that are used by this sorter.

close

public void close()

Shuts down all the threads initiated by this sorter. Also releases all previously allocated memory, if it has not yet been released by the threads, and closes and deletes all channels (removing the temporary files).

The threads are set to exit directly, but depending on their operation, it may take a while to actually happen. The sorting thread will for example not finish before the current batch is sorted. This method attempts to wait for the working thread to exit. If it is however interrupted, the method exits immediately and is not guaranteed how long the threads continue to exist and occupy resources afterwards.

指定者:

close 在接口中 Closeable

指定者:

close 在接口中 AutoCloseable

write

public void write(org.apache.flink.table.data.RowData current)
           throws IOException

抛出:

IOException

write

@VisibleForTesting
public void write(org.apache.flink.util.MutableObjectIterator<org.apache.flink.table.data.binary.BinaryRowData> iterator)
                              throws IOException

抛出:

IOException

getIterator

public org.apache.flink.util.MutableObjectIterator<org.apache.flink.table.data.binary.BinaryRowData> getIterator()
                                                                                                          throws InterruptedException

指定者:

getIterator 在接口中 org.apache.flink.runtime.operators.sort.Sorter<org.apache.flink.table.data.binary.BinaryRowData>

指定者:

getIterator 在接口中 org.apache.flink.runtime.operators.util.CloseableInputProvider<org.apache.flink.table.data.binary.BinaryRowData>

抛出:

InterruptedException

getUsedMemoryInBytes

public long getUsedMemoryInBytes()

getNumSpillFiles

public long getNumSpillFiles()

getSpillInBytes

public long getSpillInBytes()