org.apache.hadoop.hive.metastore.columnstats.aggr

Class ColumnStatsAggregator

java.lang.Object

org.apache.hadoop.hive.metastore.columnstats.aggr.ColumnStatsAggregator

Direct Known Subclasses:

BinaryColumnStatsAggregator, BooleanColumnStatsAggregator, DateColumnStatsAggregator, DecimalColumnStatsAggregator, DoubleColumnStatsAggregator, LongColumnStatsAggregator, StringColumnStatsAggregator, TimestampColumnStatsAggregator

public abstract class ColumnStatsAggregator
extends Object

Field Summary

Fields

Modifier and Type	Field and Description
double	ndvTuner The tuner controls the derivation of the NDV value when aggregating statistics from multiple partitions.
boolean	useDensityFunctionForNDVEstimation

Constructor Summary

Constructors

Constructor and Description
ColumnStatsAggregator()

Method Summary

Modifier and Type	Method and Description
abstract ColumnStatisticsObj	aggregate(List<MetaStoreServerUtils.ColStatsObjWithSourceInfo> colStatsWithSourceInfo, List<String> partNames, boolean areAllPartsFound)
protected abstract ColumnStatisticsData	initColumnStatisticsData()
protected KllHistogramEstimator	mergeHistograms(List<MetaStoreServerUtils.ColStatsObjWithSourceInfo> colStatsWithSourceInfo)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

useDensityFunctionForNDVEstimation

public boolean useDensityFunctionForNDVEstimation

ndvTuner

public double ndvTuner

The tuner controls the derivation of the NDV value when aggregating statistics from multiple partitions. It accepts values in the range [0, 1] pushing the aggregated NDV closer to the lower, or upper bound respectively.

For example, consider the aggregation of three partitions with NDV values 2, 3, and 4, respectively. The NDV lower bound is 4 (the highest among individual NDVs), and the upper bound is 9 (the sum of individual NDVs). In this case the aggregated NDV will be in the range [4, 9] touching the bounds when the tuner is equal to 0, or 1 respectively.

It is optional and concrete implementations can choose to ignore it completely.

Constructor Detail

ColumnStatsAggregator

public ColumnStatsAggregator()

Method Detail

aggregate

public abstract ColumnStatisticsObj aggregate(List<MetaStoreServerUtils.ColStatsObjWithSourceInfo> colStatsWithSourceInfo,
                                              List<String> partNames,
                                              boolean areAllPartsFound)
                                       throws MetaException

Throws:

MetaException

initColumnStatisticsData

protected abstract ColumnStatisticsData initColumnStatisticsData()

mergeHistograms

protected KllHistogramEstimator mergeHistograms(List<MetaStoreServerUtils.ColStatsObjWithSourceInfo> colStatsWithSourceInfo)