org.apache.drill.exec.physical.impl.orderedpartitioner

Class OrderedPartitionRecordBatch

java.lang.Object

org.apache.drill.exec.record.AbstractRecordBatch<OrderedPartitionSender>

org.apache.drill.exec.physical.impl.orderedpartitioner.OrderedPartitionRecordBatch

All Implemented Interfaces:

AutoCloseable, Iterable<VectorWrapper<?>>, CloseableRecordBatch, RecordBatch, VectorAccessible

public class OrderedPartitionRecordBatch
extends AbstractRecordBatch<OrderedPartitionSender>

The purpose of this operator is to generate an ordered partition, rather than a random hash partition. This could be used to do a total order sort, for example. This operator reads in a few incoming record batches, samples these batches, and stores them in the distributed cache. The samples from all the parallel-running fragments are merged, and a partition-table is built and stored in the distributed cache for use by all fragments. A new column is added to the outgoing batch, whose value is determined by where each record falls in the partition table. This column is used by PartitionSenderRootExec to determine which bucket to assign each record to.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.drill.exec.record.AbstractRecordBatch

AbstractRecordBatch.BatchState

Nested classes/interfaces inherited from interface org.apache.drill.exec.record.RecordBatch

RecordBatch.IterOutcome

Field Summary

Fields

Modifier and Type	Field and Description
protected RecordBatch	incoming
MappingSet	incomingMapping
MappingSet	mainMapping
static DistributedCache.CacheConfig<String,CachedVectorContainer>	MULTI_CACHE_CONFIG
MappingSet	partitionMapping
static DistributedCache.CacheConfig<String,CachedVectorContainer>	SINGLE_CACHE_CONFIG

Fields inherited from class org.apache.drill.exec.record.AbstractRecordBatch

container, context, oContext, popConfig, state, stats, unionTypeEnabled

Fields inherited from interface org.apache.drill.exec.record.RecordBatch

MAX_BATCH_SIZE

Constructor Summary

Constructors

Constructor and Description
OrderedPartitionRecordBatch(OrderedPartitionSender pop, RecordBatch incoming, FragmentContext context)

Method Summary

Methods

Modifier and Type	Method and Description
void	close()
protected void	doWork(VectorAccessible batch)
int	getRecordCount() Gets the number of records that are within this record.
RecordBatch.IterOutcome	innerNext()
protected void	killIncoming(boolean sendUpstream)
protected void	setupNewSchema(VectorAccessible batch) Sets up projection that will transfer all of the columns in batch, and also populate the partition column based on which partition a record falls into in the partition table

Methods inherited from class org.apache.drill.exec.record.AbstractRecordBatch

buildSchema, getContext, getOutgoingContainer, getPopConfig, getSchema, getSelectionVector2, getSelectionVector4, getValueAccessorById, getValueVectorId, getWritableBatch, iterator, kill, next, next, next

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

SINGLE_CACHE_CONFIG

public static final DistributedCache.CacheConfig<String,CachedVectorContainer> SINGLE_CACHE_CONFIG

MULTI_CACHE_CONFIG

public static final DistributedCache.CacheConfig<String,CachedVectorContainer> MULTI_CACHE_CONFIG

mainMapping

public final MappingSet mainMapping

incomingMapping

public final MappingSet incomingMapping

partitionMapping

public final MappingSet partitionMapping

incoming

protected final RecordBatch incoming

Constructor Detail

OrderedPartitionRecordBatch

public OrderedPartitionRecordBatch(OrderedPartitionSender pop,
                           RecordBatch incoming,
                           FragmentContext context)
                            throws OutOfMemoryException

Throws:

OutOfMemoryException

Method Detail

close

public void close()

Specified by:

close in interface AutoCloseable

Overrides:

close in class AbstractRecordBatch<OrderedPartitionSender>

killIncoming

protected void killIncoming(boolean sendUpstream)

Specified by:

killIncoming in class AbstractRecordBatch<OrderedPartitionSender>

innerNext

public RecordBatch.IterOutcome innerNext()

Specified by:

innerNext in class AbstractRecordBatch<OrderedPartitionSender>

getRecordCount

public int getRecordCount()

Description copied from interface: RecordBatch

Gets the number of records that are within this record.

Returns:

number of records

doWork

protected void doWork(VectorAccessible batch)

setupNewSchema

protected void setupNewSchema(VectorAccessible batch)
                       throws SchemaChangeException

Sets up projection that will transfer all of the columns in batch, and also populate the partition column based on which partition a record falls into in the partition table

Parameters:

batch -

Throws:

SchemaChangeException