org.apache.beam.sdk.coders

Class StandardCoder<T>

java.lang.Object

org.apache.beam.sdk.coders.StandardCoder<T>

All Implemented Interfaces:

Serializable, Coder<T>

Direct Known Subclasses:

AvroCoder, CoGbkResult.CoGbkResultCoder, DeterministicStandardCoder, IterableLikeCoder, KvCoder, MapCoder, NullableCoder, TimestampedValue.TimestampedValueCoder, UnionCoder

public abstract class StandardCoder<T>
extends Object
implements Coder<T>

An abstract base class to implement a Coder that defines equality, hashing, and printing via the class name and recursively using getComponents().

To extend StandardCoder, override the following methods as appropriate:

• getComponents(): the default implementation returns Coder.getCoderArguments().
• getEncodedElementByteSize(T, org.apache.beam.sdk.coders.Coder.Context) and isRegisterByteSizeObserverCheap(T, org.apache.beam.sdk.coders.Coder.Context): the default implementation encodes values to bytes and counts the bytes, which is considered expensive.
• getEncodingId() and getAllowedEncodings(): by default, the encoding id is the empty string, so only the canonical name of the subclass will be used for compatibility checks, and no other encoding ids are allowed.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.beam.sdk.coders.Coder

Coder.Context, Coder.NonDeterministicException

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	StandardCoder()

Method Summary

Modifier and Type	Method and Description
org.apache.beam.sdk.util.CloudObject	asCloudObject() Returns the CloudObject that represents this Coder.
boolean	consistentWithEquals() Returns true if this Coder is injective with respect to Object.equals(java.lang.Object).
boolean	equals(Object o)
Collection<String>	getAllowedEncodings() A collection of encodings supported by Coder.decode(java.io.InputStream, org.apache.beam.sdk.coders.Coder.Context) in addition to the encoding from Coder.getEncodingId() (which is assumed supported).
List<? extends Coder<?>>	getComponents() Returns the list of Coders that are components of this Coder.
protected long	getEncodedElementByteSize(T value, Coder.Context context) Returns the size in bytes of the encoded value using this coder.
String	getEncodingId() An identifier for the binary format written by Coder.encode(T, java.io.OutputStream, org.apache.beam.sdk.coders.Coder.Context).
int	hashCode()
boolean	isRegisterByteSizeObserverCheap(T value, Coder.Context context) Returns whether Coder.registerByteSizeObserver(T, org.apache.beam.sdk.util.common.ElementByteSizeObserver, org.apache.beam.sdk.coders.Coder.Context) cheap enough to call for every element, that is, if this Coder can calculate the byte size of the element to be coded in roughly constant time (or lazily).
void	registerByteSizeObserver(T value, org.apache.beam.sdk.util.common.ElementByteSizeObserver observer, Coder.Context context) Notifies the ElementByteSizeObserver about the byte size of the encoded value using this Coder.
Object	structuralValue(T value) Returns an object with an Object.equals() method that represents structural equality on the argument.
String	toString()
protected void	verifyDeterministic(String message, Coder<?>... coders)
protected void	verifyDeterministic(String message, Iterable<Coder<?>> coders)

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.beam.sdk.coders.Coder

decode, encode, getCoderArguments, verifyDeterministic

Constructor Detail

StandardCoder

protected StandardCoder()

Method Detail

getEncodingId

public String getEncodingId()

Description copied from interface: Coder

An identifier for the binary format written by Coder.encode(T, java.io.OutputStream, org.apache.beam.sdk.coders.Coder.Context).

This value, along with the fully qualified class name, forms an identifier for the binary format of this coder. Whenever this value changes, the new encoding is considered incompatible with the prior format: It is presumed that the prior version of the coder will be unable to correctly read the new format and the new version of the coder will be unable to correctly read the old format.

If the format is changed in a backwards-compatible way (the Coder can still accept data from the prior format), such as by adding optional fields to a Protocol Buffer or Avro definition, and you want Dataflow to understand that the new coder is compatible with the prior coder, this value must remain unchanged. It is then the responsibility of Coder.decode(java.io.InputStream, org.apache.beam.sdk.coders.Coder.Context) to correctly read data from the prior format.

Specified by:

getEncodingId in interface Coder<T>

getAllowedEncodings

public Collection<String> getAllowedEncodings()

Description copied from interface: Coder

A collection of encodings supported by Coder.decode(java.io.InputStream, org.apache.beam.sdk.coders.Coder.Context) in addition to the encoding from Coder.getEncodingId() (which is assumed supported).

This information is not currently used for any purpose. It is descriptive only, and this method is subject to change.

Specified by:

getAllowedEncodings in interface Coder<T>

See Also:

Coder.getEncodingId()

getComponents

public List<? extends Coder<?>> getComponents()

Returns the list of Coders that are components of this Coder.

equals

public boolean equals(Object o)

Overrides:

equals in class Object

Returns:

true if the two StandardCoder instances have the same class and equal components.

hashCode

public int hashCode()

Overrides:

hashCode in class Object

toString

public String toString()

Overrides:

toString in class Object

asCloudObject

public org.apache.beam.sdk.util.CloudObject asCloudObject()

Description copied from interface: Coder

Returns the CloudObject that represents this Coder.

Specified by:

asCloudObject in interface Coder<T>

isRegisterByteSizeObserverCheap

public boolean isRegisterByteSizeObserverCheap(T value,
                                               Coder.Context context)

Returns whether Coder.registerByteSizeObserver(T, org.apache.beam.sdk.util.common.ElementByteSizeObserver, org.apache.beam.sdk.coders.Coder.Context) cheap enough to call for every element, that is, if this Coder can calculate the byte size of the element to be coded in roughly constant time (or lazily).

Not intended to be called by user code, but instead by PipelineRunner implementations.

Specified by:

isRegisterByteSizeObserverCheap in interface Coder<T>

Returns:

false unless it is overridden. registerByteSizeObserver(T, org.apache.beam.sdk.util.common.ElementByteSizeObserver, org.apache.beam.sdk.coders.Coder.Context) invokes getEncodedElementByteSize(T, org.apache.beam.sdk.coders.Coder.Context) which requires re-encoding an element unless it is overridden. This is considered expensive.

getEncodedElementByteSize

protected long getEncodedElementByteSize(T value,
                                         Coder.Context context)
                                  throws Exception

Returns the size in bytes of the encoded value using this coder.

Throws:

Exception

registerByteSizeObserver

public void registerByteSizeObserver(T value,
                                     org.apache.beam.sdk.util.common.ElementByteSizeObserver observer,
                                     Coder.Context context)
                              throws Exception

Notifies the ElementByteSizeObserver about the byte size of the encoded value using this Coder.

Not intended to be called by user code, but instead by PipelineRunner implementations.

For StandardCoder subclasses, this notifies observer about the byte size of the encoded value using this coder as returned by getEncodedElementByteSize(T, org.apache.beam.sdk.coders.Coder.Context).

Specified by:

registerByteSizeObserver in interface Coder<T>

Throws:

Exception

verifyDeterministic

protected void verifyDeterministic(String message,
                                   Iterable<Coder<?>> coders)
                            throws Coder.NonDeterministicException

Throws:

Coder.NonDeterministicException

verifyDeterministic

protected void verifyDeterministic(String message,
                                   Coder<?>... coders)
                            throws Coder.NonDeterministicException

Throws:

Coder.NonDeterministicException

consistentWithEquals

public boolean consistentWithEquals()

Returns true if this Coder is injective with respect to Object.equals(java.lang.Object).

Whenever the encoded bytes of two values are equal, then the original values are equal according to Objects.equals(). Note that this is well-defined for null.

This condition is most notably false for arrays. More generally, this condition is false whenever equals() compares object identity, rather than performing a semantic/structural comparison.

Specified by:

consistentWithEquals in interface Coder<T>

Returns:

false for StandardCoder unless overridden.

structuralValue

public Object structuralValue(T value)
                       throws Exception

Description copied from interface: Coder

Returns an object with an Object.equals() method that represents structural equality on the argument.

For any two values x and y of type T, if their encoded bytes are the same, then it must be the case that structuralValue(x).equals(@code structuralValue(y).

Most notably:

• The structural value for an array coder should perform a structural comparison of the contents of the arrays, rather than the default behavior of comparing according to object identity.
• The structural value for a coder accepting null should be a proper object with an equals() method, even if the input value is null.

See also Coder.consistentWithEquals().

Specified by:

structuralValue in interface Coder<T>

Throws:

Exception