org.apache.beam.sdk.extensions.sketching

Class ApproximateDistinct.PerKeyDistinct<K,V>

java.lang.Object

org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<K,V>>,org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<K,java.lang.Long>>>

org.apache.beam.sdk.extensions.sketching.ApproximateDistinct.PerKeyDistinct<K,V>

Type Parameters:

K - type of the keys mapping the elements

V - type of the values being combined per key

All Implemented Interfaces:

java.io.Serializable, org.apache.beam.sdk.transforms.display.HasDisplayData

Enclosing class:

ApproximateDistinct

public abstract static class ApproximateDistinct.PerKeyDistinct<K,V>
extends org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<K,V>>,org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<K,java.lang.Long>>>

Implementation of ApproximateDistinct.perKey().

See Also:

Serialized Form

Field Summary

Fields inherited from class org.apache.beam.sdk.transforms.PTransform

annotations, displayData, name, resourceHints

Constructor Summary

Constructors

Constructor and Description
PerKeyDistinct()

Method Summary

Modifier and Type	Method and Description
org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<K,java.lang.Long>>	expand(org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<K,V>> input)
ApproximateDistinct.PerKeyDistinct<K,V>	withPrecision(int p) Sets the precision p.
ApproximateDistinct.PerKeyDistinct<K,V>	withSparsePrecision(int sp) Sets the sparse representation's precision sp.

Methods inherited from class org.apache.beam.sdk.transforms.PTransform

addAnnotation, compose, compose, getAdditionalInputs, getAnnotations, getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, getResourceHints, populateDisplayData, setDisplayData, setResourceHints, toString, validate, validate

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

PerKeyDistinct

public PerKeyDistinct()

Method Detail

withPrecision

public ApproximateDistinct.PerKeyDistinct<K,V> withPrecision(int p)

Sets the precision p.

Keep in mind that p cannot be lower than 4, because the estimation would be too inaccurate.

See ApproximateDistinct.precisionForRelativeError(double) and ApproximateDistinct.relativeErrorForPrecision(int) to have more information about the relationship between precision and relative error.

Parameters:

p - the precision value for the normal representation

withSparsePrecision

public ApproximateDistinct.PerKeyDistinct<K,V> withSparsePrecision(int sp)

Sets the sparse representation's precision sp.

Values above 32 are not yet supported by the AddThis version of HyperLogLog+.

Fore more information about the sparse representation, read Google's paper available here.

Parameters:

sp - the precision of HyperLogLog+' sparse representation

expand

public org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<K,java.lang.Long>> expand(org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<K,V>> input)

Specified by:

expand in class org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<K,V>>,org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<K,java.lang.Long>>>