Package org.apache.beam.sdk.extensions.combiners

Class Histogram

java.lang.Object

org.apache.beam.sdk.extensions.combiners.Histogram

public class Histogram
extends java.lang.Object

A histogram transform with a combiner that efficiently constructs linear, exponential or explicit histograms from large datasets of input data. Bucket bounds can be specified using the Histogram.BucketBounds class.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	Histogram.BoundsInclusivity	Enum for setting whether the lower (and upper) bounds of bucket intervals are inclusive or exclusive.
static class	Histogram.BucketBounds	Defines the bounds for histogram buckets.
static class	Histogram.HistogramCombineFn<T>	Combiner for calculating histograms.

Method Summary

Modifier and Type	Method	Description
static <T extends java.lang.Number> org.apache.beam.sdk.transforms.Combine.Globally<T,java.util.List<java.lang.Long>>	globally(Histogram.BucketBounds bucketBounds)	Returns a PTransform that takes a PCollection<T> and returns a PCollection<List<Long>> with a single element per window.
static <K,V extends java.lang.Number> org.apache.beam.sdk.transforms.Combine.PerKey<K,V,java.util.List<java.lang.Long>>	perKey(Histogram.BucketBounds bucketBounds)	Returns a PTransform that takes a PCollection<KV<K, V>> and returns a PCollection<KV<K, List<Long>>> that contains an output element mapping each distinct key in the input PCollection to a List.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

globally

public static <T extends java.lang.Number> org.apache.beam.sdk.transforms.Combine.Globally<T,java.util.List<java.lang.Long>> globally(Histogram.BucketBounds bucketBounds)

Returns a PTransform that takes a PCollection<T> and returns a PCollection<List<Long>> with a single element per window. The values of this list represent the number of elements within each bucket of a histogram, as defined by Histogram.BucketBounds. The first and the last elements of the list are numbers of elements in underflow and overflow buckets.

Example of use:

 PCollection<Double> pc = ...;
 PCollection<List<Long>> bucketCounts =
     pc.apply(Histogram.globally(BucketBounds.linear(1.0, 2.0, 100)));

 

Type Parameters:

T - the type of the elements in the input PCollection

Parameters:

bucketBounds - the instance of the Histogram.BucketBounds class with desired parameters of the histogram.

perKey

public static <K,V extends java.lang.Number> org.apache.beam.sdk.transforms.Combine.PerKey<K,V,java.util.List<java.lang.Long>> perKey(Histogram.BucketBounds bucketBounds)

Returns a PTransform that takes a PCollection<KV<K, V>> and returns a PCollection<KV<K, List<Long>>> that contains an output element mapping each distinct key in the input PCollection to a List. The values of this list represent the number of elements within each bucket of a histogram, as defined by Histogram.BucketBounds. The first and the last elements of the list are numbers of elements in underflow and overflow buckets.

Example of use:

 PCollection<KV<String, Integer>> pc = ...;
 PCollection<KV<String, List<Long>>> bucketCounts =
     pc.apply(Histogram.perKey(BucketBounds.linear(1.0, 2.0, 100)));

 

Type Parameters:

K - the type of the keys in the input and output PCollections

V - the type of the values in the input PCollection

Parameters:

bucketBounds - the instance of the Histogram.BucketBounds class with desired parameters of the histogram.