org.apache.beam.sdk.extensions.sketching

Class ApproximateDistinct.GloballyDistinct<InputT>

java.lang.Object

org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<InputT>,org.apache.beam.sdk.values.PCollection<java.lang.Long>>

org.apache.beam.sdk.extensions.sketching.ApproximateDistinct.GloballyDistinct<InputT>

Type Parameters:

InputT - the type of the elements in the input PCollection

All Implemented Interfaces:

java.io.Serializable, org.apache.beam.sdk.transforms.display.HasDisplayData

Enclosing class:

ApproximateDistinct

public abstract static class ApproximateDistinct.GloballyDistinct<InputT>
extends org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<InputT>,org.apache.beam.sdk.values.PCollection<java.lang.Long>>

Implementation of ApproximateDistinct.globally().

See Also:

Serialized Form

Field Summary

Fields inherited from class org.apache.beam.sdk.transforms.PTransform

name

Constructor Summary

Constructors

Constructor and Description
GloballyDistinct()

Method Summary

Modifier and Type	Method and Description
org.apache.beam.sdk.values.PCollection<java.lang.Long>	expand(org.apache.beam.sdk.values.PCollection<InputT> input)
ApproximateDistinct.GloballyDistinct<InputT>	withPrecision(int p) Sets the precision p.
ApproximateDistinct.GloballyDistinct<InputT>	withSparsePrecision(int sp) Sets the sparse representation's precision sp.

Methods inherited from class org.apache.beam.sdk.transforms.PTransform

compose, compose, getAdditionalInputs, getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, populateDisplayData, toString, validate

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

GloballyDistinct

public GloballyDistinct()

Method Detail

withPrecision

public ApproximateDistinct.GloballyDistinct<InputT> withPrecision(int p)

Sets the precision p.

Keep in mind that p cannot be lower than 4, because the estimation would be too inaccurate.

See ApproximateDistinct.precisionForRelativeError(double) and ApproximateDistinct.relativeErrorForPrecision(int) to have more information about the relationship between precision and relative error.

Parameters:

p - the precision value for the normal representation

withSparsePrecision

public ApproximateDistinct.GloballyDistinct<InputT> withSparsePrecision(int sp)

Sets the sparse representation's precision sp.

Values above 32 are not yet supported by the AddThis version of HyperLogLog+.

Fore more information about the sparse representation, read Google's paper available here.

Parameters:

sp - the precision of HyperLogLog+' sparse representation

expand

public org.apache.beam.sdk.values.PCollection<java.lang.Long> expand(org.apache.beam.sdk.values.PCollection<InputT> input)

Specified by:

expand in class org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<InputT>,org.apache.beam.sdk.values.PCollection<java.lang.Long>>