org.apache.drill.exec.util

Class MemoryAllocationUtilities

java.lang.Object

org.apache.drill.exec.util.MemoryAllocationUtilities

public class MemoryAllocationUtilities
extends Object

Constructor Summary

Constructors

Constructor and Description
MemoryAllocationUtilities()

Method Summary

Methods

Modifier and Type	Method and Description
static long	computeOperatorMemory(OptionSet optionManager, long maxAllocPerNode, int opCount) Compute per-operator memory based on the computed per-node memory, the number of operators, and the computed number of fragments (which house the operators.) Enforces a floor on the amount of memory per operator.
static long	computeQueryMemory(DrillConfig config, OptionSet optionManager, long directMemory) Per-node memory calculations based on a number of constraints.
static void	setupBufferedOpsMemoryAllocations(PhysicalPlan plan, QueryContext queryContext) Helper method to setup Memory Allocations

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

MemoryAllocationUtilities

public MemoryAllocationUtilities()

Method Detail

setupBufferedOpsMemoryAllocations

public static void setupBufferedOpsMemoryAllocations(PhysicalPlan plan,
                                     QueryContext queryContext)

Helper method to setup Memory Allocations

Plan the memory for buffered operators (the only ones that can spill in this release) based on assumptions. These assumptions are the amount of memory per node to give to each query and the number of sort operators per node.

The reason the total memory is an assumption is that we have know knowledge of the number of queries that can run, so we need the user to tell use that information by configuring the amount of memory to be assumed available to each query.

The number of sorts per node could be calculated, but we instead simply take the worst case: the maximum per-query, per-node parallization and assume that all sorts appear in all fragments — a gross oversimplification, but one that Drill has long made.

since this method can be used in multiple places adding it in this class rather than keeping it in Foreman

Parameters:

plan -

queryContext -

computeOperatorMemory

public static long computeOperatorMemory(OptionSet optionManager,
                         long maxAllocPerNode,
                         int opCount)

Compute per-operator memory based on the computed per-node memory, the number of operators, and the computed number of fragments (which house the operators.) Enforces a floor on the amount of memory per operator.

Parameters:

optionManager - system option manager

maxAllocPerNode - computed query memory per node

opCount - number of buffering operators in this query

Returns:

the per-operator memory

computeQueryMemory

public static long computeQueryMemory(DrillConfig config,
                      OptionSet optionManager,
                      long directMemory)

Per-node memory calculations based on a number of constraints.

Factored out into a separate method to allow unit testing.

Parameters:

config - Drill config

optionManager - system options

directMemory - amount of direct memory

Returns:

memory per query per node