org.apache.drill.exec.physical.impl.partitionsender

Class PartitionerDecorator

java.lang.Object

org.apache.drill.exec.physical.impl.partitionsender.PartitionerDecorator

public class PartitionerDecorator
extends Object

Decorator class to hide multiple Partitioner existence from the caller since this class involves multithreaded processing of incoming batches as well as flushing it needs special handling of OperatorStats - stats since stats are not suitable for use in multithreaded environment The algorithm to figure out processing versus wait time is based on following formula: totalWaitTime = totalAllPartitionersProcessingTime - max(sum(processingTime) by partitioner)

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected static interface	PartitionerDecorator.GeneralExecuteIface Helper interface to generalize functionality executed in the thread since it is absolutely the same for partitionBatch and flushOutgoingBatches protected is for testing purposes

Constructor Summary

Constructors

Constructor and Description
PartitionerDecorator(List<Partitioner> partitioners, OperatorStats stats, FragmentContext context)

Method Summary

Methods

Modifier and Type	Method and Description
void	clear() decorator method to call multiple Partitioners clear()
protected void	executeMethodLogic(PartitionerDecorator.GeneralExecuteIface iface) Helper to execute the different methods wrapped into same logic
void	flushOutgoingBatches(boolean isLastBatch, boolean schemaChanged) flushOutgoingBatches - decorator to call real Partitioner(s) flushOutgoingBatches
PartitionOutgoingBatch	getOutgoingBatches(int index) Helper method to get PartitionOutgoingBatch based on the index since we may have more then one Partitioner As number of Partitioners should be very small AND this method it used very rarely, so it is OK to loop in order to find right partitioner
protected List<Partitioner>	getPartitioners()
void	initialize() decorator method to call multiple Partitioners initialize()
void	partitionBatch(RecordBatch incoming) partitionBatch - decorator method to call real Partitioner(s) to process incoming batch uses either threading or not threading approach based on number Partitioners

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

PartitionerDecorator

public PartitionerDecorator(List<Partitioner> partitioners,
                    OperatorStats stats,
                    FragmentContext context)

Method Detail

partitionBatch

public void partitionBatch(RecordBatch incoming)
                    throws IOException

partitionBatch - decorator method to call real Partitioner(s) to process incoming batch uses either threading or not threading approach based on number Partitioners

Parameters:

incoming -

Throws:

IOException

flushOutgoingBatches

public void flushOutgoingBatches(boolean isLastBatch,
                        boolean schemaChanged)
                          throws IOException

flushOutgoingBatches - decorator to call real Partitioner(s) flushOutgoingBatches

Parameters:

isLastBatch -

schemaChanged -

Throws:

IOException

initialize

public void initialize()

decorator method to call multiple Partitioners initialize()

clear

public void clear()

decorator method to call multiple Partitioners clear()

getOutgoingBatches

public PartitionOutgoingBatch getOutgoingBatches(int index)

Helper method to get PartitionOutgoingBatch based on the index since we may have more then one Partitioner As number of Partitioners should be very small AND this method it used very rarely, so it is OK to loop in order to find right partitioner

Parameters:

index - - index of PartitionOutgoingBatch

Returns:

PartitionOutgoingBatch

getPartitioners

protected List<Partitioner> getPartitioners()

executeMethodLogic

protected void executeMethodLogic(PartitionerDecorator.GeneralExecuteIface iface)
                           throws IOException

Helper to execute the different methods wrapped into same logic

Parameters:

iface -

Throws:

IOException