org.apache.flink.optimizer.costs

Class DefaultCostEstimator

java.lang.Object

org.apache.flink.optimizer.costs.CostEstimator

org.apache.flink.optimizer.costs.DefaultCostEstimator

public class DefaultCostEstimator
extends CostEstimator

A default cost estimator that has access to basic size and cardinality estimates.

This estimator works with actual estimates (as far as they are available) and falls back to setting relative costs, if no estimates are available. That way, the estimator makes sure that plans with different strategies are costed differently, also in the absence of estimates. The different relative costs in the absence of estimates represent this estimator's heuristic guidance towards certain strategies.

For robustness reasons, we always assume that the whole data is shipped during a repartition step. We deviate from the typical estimate of (n - 1) / n (with n being the number of nodes), because for a parallelism of 1, that would yield a shipping of zero bytes. While this is usually correct, the runtime scheduling may still choose to move tasks to different nodes, so that we do not know that no data is shipped.

Constructor Summary

Constructors

Constructor and Description
DefaultCostEstimator()

Method Summary

Modifier and Type	Method and Description
void	addArtificialDamCost(EstimateProvider estimates, long bufferSize, Costs costs)
void	addBlockNestedLoopsCosts(EstimateProvider outerSide, EstimateProvider innerSide, long blockSize, Costs costs, int costWeight)
void	addBroadcastCost(EstimateProvider estimates, int replicationFactor, Costs costs)
void	addCachedHybridHashCosts(EstimateProvider buildSideInput, EstimateProvider probeSideInput, Costs costs, int costWeight) Calculates the costs for the cached variant of the hybrid hash join.
void	addFileInputCost(long fileSizeInBytes, Costs costs)
void	addHashPartitioningCost(EstimateProvider estimates, Costs costs)
void	addHybridHashCosts(EstimateProvider buildSideInput, EstimateProvider probeSideInput, Costs costs, int costWeight)
void	addLocalMergeCost(EstimateProvider input1, EstimateProvider input2, Costs costs, int costWeight)
void	addLocalSortCost(EstimateProvider estimates, Costs costs)
void	addRandomPartitioningCost(EstimateProvider estimates, Costs costs)
void	addRangePartitionCost(EstimateProvider estimates, Costs costs)
void	addStreamedNestedLoopsCosts(EstimateProvider outerSide, EstimateProvider innerSide, long bufferSize, Costs costs, int costWeight)

Methods inherited from class org.apache.flink.optimizer.costs.CostEstimator

costOperator

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DefaultCostEstimator

public DefaultCostEstimator()

Method Detail

addRandomPartitioningCost

public void addRandomPartitioningCost(EstimateProvider estimates,
                                      Costs costs)

Specified by:

addRandomPartitioningCost in class CostEstimator

addHashPartitioningCost

public void addHashPartitioningCost(EstimateProvider estimates,
                                    Costs costs)

Specified by:

addHashPartitioningCost in class CostEstimator

addRangePartitionCost

public void addRangePartitionCost(EstimateProvider estimates,
                                  Costs costs)

Specified by:

addRangePartitionCost in class CostEstimator

addBroadcastCost

public void addBroadcastCost(EstimateProvider estimates,
                             int replicationFactor,
                             Costs costs)

Specified by:

addBroadcastCost in class CostEstimator

addFileInputCost

public void addFileInputCost(long fileSizeInBytes,
                             Costs costs)

Specified by:

addFileInputCost in class CostEstimator

addLocalSortCost

public void addLocalSortCost(EstimateProvider estimates,
                             Costs costs)

Specified by:

addLocalSortCost in class CostEstimator

addLocalMergeCost

public void addLocalMergeCost(EstimateProvider input1,
                              EstimateProvider input2,
                              Costs costs,
                              int costWeight)

Specified by:

addLocalMergeCost in class CostEstimator

addHybridHashCosts

public void addHybridHashCosts(EstimateProvider buildSideInput,
                               EstimateProvider probeSideInput,
                               Costs costs,
                               int costWeight)

Specified by:

addHybridHashCosts in class CostEstimator

addCachedHybridHashCosts

public void addCachedHybridHashCosts(EstimateProvider buildSideInput,
                                     EstimateProvider probeSideInput,
                                     Costs costs,
                                     int costWeight)

Calculates the costs for the cached variant of the hybrid hash join. We are assuming by default that half of the cached hash table fit into memory.

Specified by:

addCachedHybridHashCosts in class CostEstimator

addStreamedNestedLoopsCosts

public void addStreamedNestedLoopsCosts(EstimateProvider outerSide,
                                        EstimateProvider innerSide,
                                        long bufferSize,
                                        Costs costs,
                                        int costWeight)

Specified by:

addStreamedNestedLoopsCosts in class CostEstimator

addBlockNestedLoopsCosts

public void addBlockNestedLoopsCosts(EstimateProvider outerSide,
                                     EstimateProvider innerSide,
                                     long blockSize,
                                     Costs costs,
                                     int costWeight)

Specified by:

addBlockNestedLoopsCosts in class CostEstimator

addArtificialDamCost

public void addArtificialDamCost(EstimateProvider estimates,
                                 long bufferSize,
                                 Costs costs)

Specified by:

addArtificialDamCost in class CostEstimator