org.apache.beam.sdk.extensions.sorter

Class SortValues<PrimaryKeyT,SecondaryKeyT,ValueT>

java.lang.Object

org.apache.beam.sdk.transforms.PTransform<PCollection<KV<PrimaryKeyT,Iterable<KV<SecondaryKeyT,ValueT>>>>,PCollection<KV<PrimaryKeyT,Iterable<KV<SecondaryKeyT,ValueT>>>>>

org.apache.beam.sdk.extensions.sorter.SortValues<PrimaryKeyT,SecondaryKeyT,ValueT>

All Implemented Interfaces:

Serializable, HasDisplayData

public class SortValues<PrimaryKeyT,SecondaryKeyT,ValueT>
extends PTransform<PCollection<KV<PrimaryKeyT,Iterable<KV<SecondaryKeyT,ValueT>>>>,PCollection<KV<PrimaryKeyT,Iterable<KV<SecondaryKeyT,ValueT>>>>>

SortValues<PrimaryKeyT, SecondaryKeyT, ValueT> takes a PCollection<KV<PrimaryKeyT, Iterable<KV<SecondaryKeyT, ValueT>>>> with elements consisting of a primary key and iterables over <secondary key, value> pairs, and returns a PCollection<KV<PrimaryKeyT, Iterable<KV<SecondaryKeyT, ValueT>>> of the same elements but with values sorted by a secondary key.

This transform ignores the primary key, there is no guarantee of any relationship between elements with different primary keys. The primary key is explicit here only because this transform is typically used on a result of a GroupByKey transform.

This transform sorts by lexicographic comparison of the byte representations of the secondary keys and may write secondary key-value pairs to disk. In order to retrieve the byte representations it requires the input PCollection to use a KvCoder for its input, an IterableCoder for its input values and a KvCoder for its secondary key-value pairs.

See Also:

Serialized Form

Field Summary

Fields inherited from class org.apache.beam.sdk.transforms.PTransform

name

Method Summary

Modifier and Type	Method and Description
static <PrimaryKeyT,SecondaryKeyT,ValueT> SortValues<PrimaryKeyT,SecondaryKeyT,ValueT>	create(BufferedExternalSorter.Options sorterOptions) Returns a SortValues<PrimaryKeyT, SecondaryKeyT, ValueT> PTransform.
PCollection<KV<PrimaryKeyT,Iterable<KV<SecondaryKeyT,ValueT>>>>	expand(PCollection<KV<PrimaryKeyT,Iterable<KV<SecondaryKeyT,ValueT>>>> input)
protected Coder<KV<PrimaryKeyT,Iterable<KV<SecondaryKeyT,ValueT>>>>	getDefaultOutputCoder(PCollection<KV<PrimaryKeyT,Iterable<KV<SecondaryKeyT,ValueT>>>> input)

Methods inherited from class org.apache.beam.sdk.transforms.PTransform

getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, populateDisplayData, toString, validate

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Method Detail

create

public static <PrimaryKeyT,SecondaryKeyT,ValueT> SortValues<PrimaryKeyT,SecondaryKeyT,ValueT> create(BufferedExternalSorter.Options sorterOptions)

Returns a SortValues<PrimaryKeyT, SecondaryKeyT, ValueT> PTransform.

Type Parameters:

PrimaryKeyT - the type of the primary keys of the input and output PCollections

SecondaryKeyT - the type of the secondary (sort) keys of the input and output PCollections

ValueT - the type of the values of the input and output PCollections

expand

public PCollection<KV<PrimaryKeyT,Iterable<KV<SecondaryKeyT,ValueT>>>> expand(PCollection<KV<PrimaryKeyT,Iterable<KV<SecondaryKeyT,ValueT>>>> input)

Specified by:

expand in class PTransform<PCollection<KV<PrimaryKeyT,Iterable<KV<SecondaryKeyT,ValueT>>>>,PCollection<KV<PrimaryKeyT,Iterable<KV<SecondaryKeyT,ValueT>>>>>

getDefaultOutputCoder

protected Coder<KV<PrimaryKeyT,Iterable<KV<SecondaryKeyT,ValueT>>>> getDefaultOutputCoder(PCollection<KV<PrimaryKeyT,Iterable<KV<SecondaryKeyT,ValueT>>>> input)

Overrides:

getDefaultOutputCoder in class PTransform<PCollection<KV<PrimaryKeyT,Iterable<KV<SecondaryKeyT,ValueT>>>>,PCollection<KV<PrimaryKeyT,Iterable<KV<SecondaryKeyT,ValueT>>>>>