org.apache.beam.sdk.extensions.sketching

Class SketchFrequencies.CountMinSketchFn<InputT>

java.lang.Object

org.apache.beam.sdk.transforms.Combine.CombineFn<InputT,SketchFrequencies.Sketch<InputT>,SketchFrequencies.Sketch<InputT>>

org.apache.beam.sdk.extensions.sketching.SketchFrequencies.CountMinSketchFn<InputT>

Type Parameters:

InputT - the type of the elements in the input PCollection

All Implemented Interfaces:

java.io.Serializable, org.apache.beam.sdk.transforms.CombineFnBase.GlobalCombineFn<InputT,SketchFrequencies.Sketch<InputT>,SketchFrequencies.Sketch<InputT>>, org.apache.beam.sdk.transforms.display.HasDisplayData

Enclosing class:

SketchFrequencies

public static class SketchFrequencies.CountMinSketchFn<InputT>
extends org.apache.beam.sdk.transforms.Combine.CombineFn<InputT,SketchFrequencies.Sketch<InputT>,SketchFrequencies.Sketch<InputT>>

Implements the Combine.CombineFn of SketchFrequencies transforms.

See Also:

Serialized Form

Method Summary

Modifier and Type	Method and Description
SketchFrequencies.Sketch<InputT>	addInput(SketchFrequencies.Sketch<InputT> accumulator, InputT element)
static <InputT> SketchFrequencies.CountMinSketchFn<InputT>	create(org.apache.beam.sdk.coders.Coder<InputT> coder) Returns a SketchFrequencies.CountMinSketchFn combiner with the given input coder.
SketchFrequencies.Sketch<InputT>	createAccumulator()
SketchFrequencies.Sketch<InputT>	extractOutput(SketchFrequencies.Sketch<InputT> accumulator) Output the whole structure so it can be queried, reused or stored easily.
org.apache.beam.sdk.coders.Coder<SketchFrequencies.Sketch<InputT>>	getAccumulatorCoder(org.apache.beam.sdk.coders.CoderRegistry registry, org.apache.beam.sdk.coders.Coder inputCoder)
SketchFrequencies.Sketch<InputT>	mergeAccumulators(java.lang.Iterable<SketchFrequencies.Sketch<InputT>> accumulators)
void	populateDisplayData(org.apache.beam.sdk.transforms.display.DisplayData.Builder builder)
SketchFrequencies.CountMinSketchFn<InputT>	withAccuracy(double epsilon, double confidence) Returns a new SketchFrequencies.CountMinSketchFn combiner with new precision accuracy parameters epsilon and confidence.

Methods inherited from class org.apache.beam.sdk.transforms.Combine.CombineFn

apply, compact, defaultValue, getInputType, getOutputType

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

create

public static <InputT> SketchFrequencies.CountMinSketchFn<InputT> create(org.apache.beam.sdk.coders.Coder<InputT> coder)

Returns a SketchFrequencies.CountMinSketchFn combiner with the given input coder.
Warning : the coder must be deterministic.

Parameters:

coder - the coder that encodes the elements' type

withAccuracy

public SketchFrequencies.CountMinSketchFn<InputT> withAccuracy(double epsilon,
                                                               double confidence)

Returns a new SketchFrequencies.CountMinSketchFn combiner with new precision accuracy parameters epsilon and confidence.

Keep in mind that the lower the epsilon value, the greater the width, and the greater the confidence, the greater the depth.

Parameters:

epsilon - the error relative to the total number of distinct elements

confidence - the confidence in the result to not exceed the relative error

createAccumulator

public SketchFrequencies.Sketch<InputT> createAccumulator()

Specified by:

createAccumulator in class org.apache.beam.sdk.transforms.Combine.CombineFn<InputT,SketchFrequencies.Sketch<InputT>,SketchFrequencies.Sketch<InputT>>

addInput

public SketchFrequencies.Sketch<InputT> addInput(SketchFrequencies.Sketch<InputT> accumulator,
                                                 InputT element)

Specified by:

addInput in class org.apache.beam.sdk.transforms.Combine.CombineFn<InputT,SketchFrequencies.Sketch<InputT>,SketchFrequencies.Sketch<InputT>>

mergeAccumulators

public SketchFrequencies.Sketch<InputT> mergeAccumulators(java.lang.Iterable<SketchFrequencies.Sketch<InputT>> accumulators)

Specified by:

mergeAccumulators in class org.apache.beam.sdk.transforms.Combine.CombineFn<InputT,SketchFrequencies.Sketch<InputT>,SketchFrequencies.Sketch<InputT>>

extractOutput

public SketchFrequencies.Sketch<InputT> extractOutput(SketchFrequencies.Sketch<InputT> accumulator)

Output the whole structure so it can be queried, reused or stored easily.

Specified by:

extractOutput in class org.apache.beam.sdk.transforms.Combine.CombineFn<InputT,SketchFrequencies.Sketch<InputT>,SketchFrequencies.Sketch<InputT>>

getAccumulatorCoder

public org.apache.beam.sdk.coders.Coder<SketchFrequencies.Sketch<InputT>> getAccumulatorCoder(org.apache.beam.sdk.coders.CoderRegistry registry,
                                                                                              org.apache.beam.sdk.coders.Coder inputCoder)

Specified by:

getAccumulatorCoder in interface org.apache.beam.sdk.transforms.CombineFnBase.GlobalCombineFn<InputT,SketchFrequencies.Sketch<InputT>,SketchFrequencies.Sketch<InputT>>

Overrides:

getAccumulatorCoder in class org.apache.beam.sdk.transforms.Combine.CombineFn<InputT,SketchFrequencies.Sketch<InputT>,SketchFrequencies.Sketch<InputT>>

populateDisplayData

public void populateDisplayData(org.apache.beam.sdk.transforms.display.DisplayData.Builder builder)

Specified by:

populateDisplayData in interface org.apache.beam.sdk.transforms.display.HasDisplayData

Overrides:

populateDisplayData in class org.apache.beam.sdk.transforms.Combine.CombineFn<InputT,SketchFrequencies.Sketch<InputT>,SketchFrequencies.Sketch<InputT>>