org.apache.beam.sdk.extensions.sketching

Class SketchFrequencies.CountMinSketchFn<InputT>

java.lang.Object

org.apache.beam.sdk.transforms.Combine.CombineFn<InputT,SketchFrequencies.Sketch<InputT>,SketchFrequencies.Sketch<InputT>>

org.apache.beam.sdk.extensions.sketching.SketchFrequencies.CountMinSketchFn<InputT>

Type Parameters:

InputT - the type of the elements in the input PCollection

All Implemented Interfaces:

Serializable, CombineFnBase.GlobalCombineFn<InputT,SketchFrequencies.Sketch<InputT>,SketchFrequencies.Sketch<InputT>>, HasDisplayData

Enclosing class:

SketchFrequencies

public static class SketchFrequencies.CountMinSketchFn<InputT>
extends Combine.CombineFn<InputT,SketchFrequencies.Sketch<InputT>,SketchFrequencies.Sketch<InputT>>

Implements the Combine.CombineFn of SketchFrequencies transforms.

See Also:

Serialized Form

Method Summary

Modifier and Type	Method and Description
SketchFrequencies.Sketch<InputT>	addInput(SketchFrequencies.Sketch<InputT> accumulator, InputT element)
static <InputT> SketchFrequencies.CountMinSketchFn<InputT>	create(Coder<InputT> coder) Returns an SketchFrequencies.CountMinSketchFn combiner with the given input coder.
SketchFrequencies.Sketch<InputT>	createAccumulator()
SketchFrequencies.Sketch<InputT>	extractOutput(SketchFrequencies.Sketch<InputT> accumulator) Output the whole structure so it can be queried, reused or stored easily.
Coder<SketchFrequencies.Sketch<InputT>>	getAccumulatorCoder(CoderRegistry registry, Coder inputCoder)
SketchFrequencies.Sketch<InputT>	mergeAccumulators(Iterable<SketchFrequencies.Sketch<InputT>> accumulators)
void	populateDisplayData(DisplayData.Builder builder)
SketchFrequencies.CountMinSketchFn<InputT>	withAccuracy(double epsilon, double confidence) Returns a new SketchFrequencies.CountMinSketchFn combiner with new precision accuracy parameters epsilon and confidence.

Methods inherited from class org.apache.beam.sdk.transforms.Combine.CombineFn

apply, compact, defaultValue, getOutputType

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

create

public static <InputT> SketchFrequencies.CountMinSketchFn<InputT> create(Coder<InputT> coder)

Returns an SketchFrequencies.CountMinSketchFn combiner with the given input coder.
Warning : the coder must be deterministic.

Parameters:

coder - the coder that encodes the elements' type

withAccuracy

public SketchFrequencies.CountMinSketchFn<InputT> withAccuracy(double epsilon,
                                                               double confidence)

Returns a new SketchFrequencies.CountMinSketchFn combiner with new precision accuracy parameters epsilon and confidence.

Keep in mind that the lower the epsilon value, the greater the width, and the greater the confidence, the greater the depth.

Parameters:

epsilon - the error relative to the total number of distinct elements

confidence - the confidence in the result to not exceed the relative error

createAccumulator

public SketchFrequencies.Sketch<InputT> createAccumulator()

Specified by:

createAccumulator in class Combine.CombineFn<InputT,SketchFrequencies.Sketch<InputT>,SketchFrequencies.Sketch<InputT>>

addInput

public SketchFrequencies.Sketch<InputT> addInput(SketchFrequencies.Sketch<InputT> accumulator,
                                                 InputT element)

Specified by:

addInput in class Combine.CombineFn<InputT,SketchFrequencies.Sketch<InputT>,SketchFrequencies.Sketch<InputT>>

mergeAccumulators

public SketchFrequencies.Sketch<InputT> mergeAccumulators(Iterable<SketchFrequencies.Sketch<InputT>> accumulators)

Specified by:

mergeAccumulators in class Combine.CombineFn<InputT,SketchFrequencies.Sketch<InputT>,SketchFrequencies.Sketch<InputT>>

extractOutput

public SketchFrequencies.Sketch<InputT> extractOutput(SketchFrequencies.Sketch<InputT> accumulator)

Output the whole structure so it can be queried, reused or stored easily.

Specified by:

extractOutput in class Combine.CombineFn<InputT,SketchFrequencies.Sketch<InputT>,SketchFrequencies.Sketch<InputT>>

getAccumulatorCoder

public Coder<SketchFrequencies.Sketch<InputT>> getAccumulatorCoder(CoderRegistry registry,
                                                                   Coder inputCoder)

Specified by:

getAccumulatorCoder in interface CombineFnBase.GlobalCombineFn<InputT,SketchFrequencies.Sketch<InputT>,SketchFrequencies.Sketch<InputT>>

Overrides:

getAccumulatorCoder in class Combine.CombineFn<InputT,SketchFrequencies.Sketch<InputT>,SketchFrequencies.Sketch<InputT>>

populateDisplayData

public void populateDisplayData(DisplayData.Builder builder)

Specified by:

populateDisplayData in interface HasDisplayData

Overrides:

populateDisplayData in class Combine.CombineFn<InputT,SketchFrequencies.Sketch<InputT>,SketchFrequencies.Sketch<InputT>>