public static class CombineFns.ComposedKeyedCombineFn<DataT,K> extends Combine.KeyedCombineFn<K,DataT,Object[],CombineFns.CoCombineResult>
Combine.KeyedCombineFn that applies multiple KeyedCombineFns.
For each Combine.KeyedCombineFn it extracts inputs from DataT with
the extractInputFn and combines them,
and then it outputs each combined value with a TupleTag to a
CombineFns.CoCombineResult.
| Modifier and Type | Method and Description |
|---|---|
Object[] |
addInput(K key,
Object[] accumulator,
DataT value)
Adds the given input value to the given accumulator, returning the new accumulator value.
|
Object[] |
compact(K key,
Object[] accumulator)
Returns an accumulator that represents the same logical value as the
input accumulator, but may have a more compact representation.
|
Object[] |
createAccumulator(K key)
Returns a new, mutable accumulator value representing the accumulation of zero input values.
|
CombineFns.CoCombineResult |
extractOutput(K key,
Object[] accumulator)
Returns the output value that is the result of combining all
the input values represented by the given accumulator.
|
TypeVariable<?> |
getAccumTVariable()
Returns the
TypeVariable of AccumT. |
Coder<Object[]> |
getAccumulatorCoder(CoderRegistry registry,
Coder<K> keyCoder,
Coder<DataT> dataCoder)
Returns the
Coder to use for accumulator AccumT
values, or null if it is not able to be inferred. |
Coder<OutputT> |
getDefaultOutputCoder(CoderRegistry registry,
Coder<K> keyCoder,
Coder<InputT> inputCoder)
Returns the
Coder to use by default for output
OutputT values, or null if it is not able to be inferred. |
TypeVariable<?> |
getInputTVariable()
Returns the
TypeVariable of InputT. |
TypeVariable<?> |
getKTypeVariable()
Returns the
TypeVariable of K. |
TypeVariable<?> |
getOutputTVariable()
Returns the
TypeVariable of OutputT. |
Object[] |
mergeAccumulators(K key,
Iterable<Object[]> accumulators)
Returns an accumulator representing the accumulation of all the
input values accumulated in the merging accumulators.
|
void |
populateDisplayData(DisplayData.Builder builder)
Register display data for the given transform or component.
|
<InputT,OutputT> |
with(SimpleFunction<DataT,InputT> extractInputFn,
Combine.CombineFn<InputT,?,OutputT> keyedCombineFn,
TupleTag<OutputT> outputTag)
Returns a
CombineFns.ComposedKeyedCombineFn with an additional Combine.CombineFn. |
<InputT,OutputT> |
with(SimpleFunction<DataT,InputT> extractInputFn,
Combine.KeyedCombineFn<K,InputT,?,OutputT> keyedCombineFn,
TupleTag<OutputT> outputTag)
Returns a
CombineFns.ComposedKeyedCombineFn with an additional Combine.KeyedCombineFn. |
<InputT,OutputT> |
with(SimpleFunction<DataT,InputT> extractInputFn,
CombineWithContext.CombineFnWithContext<InputT,?,OutputT> keyedCombineFn,
TupleTag<OutputT> outputTag)
Returns a
CombineFns.ComposedKeyedCombineFnWithContext with an additional
CombineWithContext.CombineFnWithContext. |
<InputT,OutputT> |
with(SimpleFunction<DataT,InputT> extractInputFn,
CombineWithContext.KeyedCombineFnWithContext<K,InputT,?,OutputT> keyedCombineFn,
TupleTag<OutputT> outputTag)
Returns a
CombineFns.ComposedKeyedCombineFnWithContext with an additional
CombineWithContext.KeyedCombineFnWithContext. |
apply, forKeypublic <InputT,OutputT> CombineFns.ComposedKeyedCombineFn<DataT,K> with(SimpleFunction<DataT,InputT> extractInputFn, Combine.KeyedCombineFn<K,InputT,?,OutputT> keyedCombineFn, TupleTag<OutputT> outputTag)
CombineFns.ComposedKeyedCombineFn with an additional Combine.KeyedCombineFn.public <InputT,OutputT> CombineFns.ComposedKeyedCombineFnWithContext<DataT,K> with(SimpleFunction<DataT,InputT> extractInputFn, CombineWithContext.KeyedCombineFnWithContext<K,InputT,?,OutputT> keyedCombineFn, TupleTag<OutputT> outputTag)
CombineFns.ComposedKeyedCombineFnWithContext with an additional
CombineWithContext.KeyedCombineFnWithContext.public <InputT,OutputT> CombineFns.ComposedKeyedCombineFn<DataT,K> with(SimpleFunction<DataT,InputT> extractInputFn, Combine.CombineFn<InputT,?,OutputT> keyedCombineFn, TupleTag<OutputT> outputTag)
CombineFns.ComposedKeyedCombineFn with an additional Combine.CombineFn.public <InputT,OutputT> CombineFns.ComposedKeyedCombineFnWithContext<DataT,K> with(SimpleFunction<DataT,InputT> extractInputFn, CombineWithContext.CombineFnWithContext<InputT,?,OutputT> keyedCombineFn, TupleTag<OutputT> outputTag)
CombineFns.ComposedKeyedCombineFnWithContext with an additional
CombineWithContext.CombineFnWithContext.public Object[] createAccumulator(K key)
Combine.KeyedCombineFncreateAccumulator in class Combine.KeyedCombineFn<K,DataT,Object[],CombineFns.CoCombineResult>key - the key that all the accumulated values using the
accumulator are associated withpublic Object[] addInput(K key, Object[] accumulator, DataT value)
Combine.KeyedCombineFnFor efficiency, the input accumulator may be modified and returned.
addInput in class Combine.KeyedCombineFn<K,DataT,Object[],CombineFns.CoCombineResult>key - the key that all the accumulated values using the
accumulator are associated withpublic Object[] mergeAccumulators(K key, Iterable<Object[]> accumulators)
Combine.KeyedCombineFnMay modify any of the argument accumulators. May return a fresh accumulator, or may return one of the (modified) argument accumulators.
mergeAccumulators in class Combine.KeyedCombineFn<K,DataT,Object[],CombineFns.CoCombineResult>key - the key that all the accumulators are associated
withpublic CombineFns.CoCombineResult extractOutput(K key, Object[] accumulator)
Combine.KeyedCombineFnextractOutput in class Combine.KeyedCombineFn<K,DataT,Object[],CombineFns.CoCombineResult>key - the key that all the accumulated values using the
accumulator are associated withpublic Object[] compact(K key, Object[] accumulator)
Combine.KeyedCombineFnFor most CombineFns this would be a no-op, but should be overridden by CombineFns that (for example) buffer up elements and combine them in batches.
For efficiency, the input accumulator may be modified and returned.
By default returns the original accumulator.
compact in class Combine.KeyedCombineFn<K,DataT,Object[],CombineFns.CoCombineResult>public Coder<Object[]> getAccumulatorCoder(CoderRegistry registry, Coder<K> keyCoder, Coder<DataT> dataCoder) throws CannotProvideCoderException
CombineFnBase.PerKeyCombineFnCoder to use for accumulator AccumT
values, or null if it is not able to be inferred.
By default, uses the knowledge of the Coder being
used for K keys and input InputT values and the
enclosing Pipeline's CoderRegistry to try to
infer the Coder for AccumT values.
This is the Coder used to send data through a communication-intensive shuffle step, so a compact and efficient representation may have significant performance benefits.
getAccumulatorCoder in interface CombineFnBase.PerKeyCombineFn<K,DataT,Object[],CombineFns.CoCombineResult>CannotProvideCoderExceptionpublic void populateDisplayData(DisplayData.Builder builder)
populateDisplayData(DisplayData.Builder) is invoked by Pipeline runners to collect
display data via DisplayData.from(HasDisplayData). Implementations may call
super.populateDisplayData(builder) in order to register display data in the current
namespace, but should otherwise use subcomponent.populateDisplayData(builder) to use
the namespace of the subcomponent.
By default, does not register any display data. Implementors may override this method to provide their own display data.
populateDisplayData in interface HasDisplayDatabuilder - The builder to populate with display data.HasDisplayDatapublic Coder<OutputT> getDefaultOutputCoder(CoderRegistry registry, Coder<K> keyCoder, Coder<InputT> inputCoder) throws CannotProvideCoderException
CombineFnBase.PerKeyCombineFnCoder to use by default for output
OutputT values, or null if it is not able to be inferred.
By default, uses the knowledge of the Coder being
used for K keys and input InputT values and the
enclosing Pipeline's CoderRegistry to try to
infer the Coder for OutputT values.
getDefaultOutputCoder in interface CombineFnBase.PerKeyCombineFn<K,InputT,AccumT,OutputT>CannotProvideCoderExceptionpublic TypeVariable<?> getKTypeVariable()
TypeVariable of K.public TypeVariable<?> getInputTVariable()
TypeVariable of InputT.public TypeVariable<?> getAccumTVariable()
TypeVariable of AccumT.public TypeVariable<?> getOutputTVariable()
TypeVariable of OutputT.