org.apache.flink.api.common.typeutils.base.array

Class PrimitiveArrayComparator<T,C extends BasicTypeComparator>

java.lang.Object

org.apache.flink.api.common.typeutils.TypeComparator<T>

org.apache.flink.api.common.typeutils.base.array.PrimitiveArrayComparator<T,C>

All Implemented Interfaces:

Serializable

Direct Known Subclasses:

BooleanPrimitiveArrayComparator, BytePrimitiveArrayComparator, CharPrimitiveArrayComparator, DoublePrimitiveArrayComparator, FloatPrimitiveArrayComparator, IntPrimitiveArrayComparator, LongPrimitiveArrayComparator, ShortPrimitiveArrayComparator

@Internal
public abstract class PrimitiveArrayComparator<T,C extends BasicTypeComparator>
extends TypeComparator<T>

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
protected boolean	ascending
protected C	comparator
protected T	reference

Constructor Summary

Constructors

Constructor and Description
PrimitiveArrayComparator(boolean ascending, C comparator)

Method Summary

Methods

Modifier and Type	Method and Description
int	compareSerialized(DataInputView firstSource, DataInputView secondSource) Compares two records in serialized form.
int	compareToReference(TypeComparator<T> referencedComparator) This method compares the element that has been set as reference in this type accessor, to the element set as reference in the given type accessor.
boolean	equalToReference(T candidate) Checks, whether the given element is equal to the element that has been set as the comparison reference in this comparator instance.
int	extractKeys(Object record, Object[] target, int index) Extracts the key fields from a record.
TypeComparator[]	getFlatComparators() Get the field comparators.
int	getNormalizeKeyLen() Gets the number of bytes that the normalized key would maximally take.
boolean	invertNormalizedKey() Flag whether normalized key comparisons should be inverted key should be interpreted inverted, i.e.
boolean	isNormalizedKeyPrefixOnly(int keyBytes) Checks, whether the given number of bytes for a normalized is only a prefix to determine the order of elements of the data type for which this comparator provides the comparison methods.
void	putNormalizedKey(T record, MemorySegment target, int offset, int numBytes) Writes a normalized key for the given record into the target byte array, starting at the specified position and writing exactly the given number of bytes.
T	readWithKeyDenormalization(T reuse, DataInputView source) Reads the record back while de-normalizing the key fields.
void	setReference(T toCompare) Sets the given element as the comparison reference for future calls to TypeComparator.equalToReference(Object) and TypeComparator.compareToReference(TypeComparator).
boolean	supportsNormalizedKey() Checks whether the data type supports the creation of a normalized key for comparison.
boolean	supportsSerializationWithKeyNormalization() Check whether this comparator supports to serialize the record in a format that replaces its keys by a normalized key.
void	writeWithKeyNormalization(T record, DataOutputView target) Writes the record in such a fashion that all keys are normalizing and at the beginning of the serialized data.

Methods inherited from class org.apache.flink.api.common.typeutils.TypeComparator

compare, compareAgainstReference, duplicate, hash, supportsCompareAgainstReference

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

ascending

protected final boolean ascending

reference

protected transient T reference

comparator

protected final C extends BasicTypeComparator comparator

Constructor Detail

PrimitiveArrayComparator

public PrimitiveArrayComparator(boolean ascending,
                        C comparator)

Method Detail

setReference

public void setReference(T toCompare)

Description copied from class: TypeComparator

Sets the given element as the comparison reference for future calls to TypeComparator.equalToReference(Object) and TypeComparator.compareToReference(TypeComparator). This method must set the given element into this comparator instance's state. If the comparison happens on a subset of the fields from the record, this method may extract those fields.

A typical example for checking the equality of two elements is the following:

E e1 = ...;
 E e2 = ...;
 
 TypeComparator<E> acc = ...;
 
 acc.setReference(e1);
 boolean equal = acc.equalToReference(e2);
 

The rational behind this method is that elements are typically compared using certain features that are extracted from them, (such de-serializing as a subset of fields). When setting the reference, this extraction happens. The extraction needs happen only once per element, even though an element is often compared to multiple other elements, such as when finding equal elements in the process of grouping the elements.

Specified by:

setReference in class TypeComparator<T>

Parameters:

toCompare - The element to set as the comparison reference.

equalToReference

public boolean equalToReference(T candidate)

Description copied from class: TypeComparator

Checks, whether the given element is equal to the element that has been set as the comparison reference in this comparator instance.

Specified by:

equalToReference in class TypeComparator<T>

Parameters:

candidate - The candidate to check.

Returns:

True, if the element is equal to the comparison reference, false otherwise.

See Also:

TypeComparator.setReference(Object)

compareToReference

public int compareToReference(TypeComparator<T> referencedComparator)

Description copied from class: TypeComparator

This method compares the element that has been set as reference in this type accessor, to the element set as reference in the given type accessor. Similar to comparing two elements e1 and e2 via a comparator, this method can be used the following way.

E e1 = ...;
 E e2 = ...;
 
 TypeComparator<E> acc1 = ...;
 TypeComparator<E> acc2 = ...;
 
 acc1.setReference(e1);
 acc2.setReference(e2);
 
 int comp = acc1.compareToReference(acc2);
 

The rational behind this method is that elements are typically compared using certain features that are extracted from them, (such de-serializing as a subset of fields). When setting the reference, this extraction happens. The extraction needs happen only once per element, even though an element is typically compared to many other elements when establishing a sorted order. The actual comparison performed by this method may be very cheap, as it happens on the extracted features.

Specified by:

compareToReference in class TypeComparator<T>

Parameters:

referencedComparator - The type accessors where the element for comparison has been set as reference.

Returns:

A value smaller than zero, if the reference value of referencedAccessors is smaller than the reference value of this type accessor; a value greater than zero, if it is larger; zero, if both are equal.

See Also:

TypeComparator.setReference(Object)

compareSerialized

public int compareSerialized(DataInputView firstSource,
                    DataInputView secondSource)
                      throws IOException

Description copied from class: TypeComparator

Compares two records in serialized form. The return value indicates the order of the two in the same way as defined by Comparator.compare(Object, Object).

This method may de-serialize the records or compare them directly based on their binary representation.

Specified by:

compareSerialized in class TypeComparator<T>

Parameters:

firstSource - The input view containing the first record.

secondSource - The input view containing the second record.

Returns:

An integer defining the oder among the objects in the same way as Comparator.compare(Object, Object).

Throws:

IOException - Thrown, if any of the input views raised an exception when reading the records.

See Also:

Comparator.compare(Object, Object)

extractKeys

public int extractKeys(Object record,
              Object[] target,
              int index)

Description copied from class: TypeComparator

Extracts the key fields from a record. This is for use by the PairComparator to provide interoperability between different record types. Note, that at least one key should be extracted.

Specified by:

extractKeys in class TypeComparator<T>

Parameters:

record - The record that contains the key(s)

target - The array to write the key(s) into.

index - The offset of the target array to start writing into.

Returns:

the number of keys added to target.

getFlatComparators

public TypeComparator[] getFlatComparators()

Description copied from class: TypeComparator

Get the field comparators. This is used together with TypeComparator.extractKeys(Object, Object[], int) to provide interoperability between different record types. Note, that this should return at least one Comparator and that the number of Comparators must match the number of extracted keys.

Specified by:

getFlatComparators in class TypeComparator<T>

Returns:

An Array of Comparators for the extracted keys.

supportsNormalizedKey

public boolean supportsNormalizedKey()

Description copied from class: TypeComparator

Checks whether the data type supports the creation of a normalized key for comparison.

Specified by:

supportsNormalizedKey in class TypeComparator<T>

Returns:

True, if the data type supports the creation of a normalized key for comparison, false otherwise.

supportsSerializationWithKeyNormalization

public boolean supportsSerializationWithKeyNormalization()

Description copied from class: TypeComparator

Check whether this comparator supports to serialize the record in a format that replaces its keys by a normalized key.

Specified by:

supportsSerializationWithKeyNormalization in class TypeComparator<T>

Returns:

True, if the comparator supports that specific form of serialization, false if not.

getNormalizeKeyLen

public int getNormalizeKeyLen()

Description copied from class: TypeComparator

Gets the number of bytes that the normalized key would maximally take. A value of Integer.MAX_VALUE is interpreted as infinite.

Specified by:

getNormalizeKeyLen in class TypeComparator<T>

Returns:

The number of bytes that the normalized key would maximally take.

isNormalizedKeyPrefixOnly

public boolean isNormalizedKeyPrefixOnly(int keyBytes)

Description copied from class: TypeComparator

Checks, whether the given number of bytes for a normalized is only a prefix to determine the order of elements of the data type for which this comparator provides the comparison methods. For example, if the data type is ordered with respect to an integer value it contains, then this method would return true, if the number of key bytes is smaller than four.

Specified by:

isNormalizedKeyPrefixOnly in class TypeComparator<T>

Returns:

True, if the given number of bytes is only a prefix, false otherwise.

putNormalizedKey

public void putNormalizedKey(T record,
                    MemorySegment target,
                    int offset,
                    int numBytes)

Description copied from class: TypeComparator

Writes a normalized key for the given record into the target byte array, starting at the specified position and writing exactly the given number of bytes. Note that the comparison of the bytes is treating the bytes as unsigned bytes: int byteI = bytes[i] & 0xFF;

If the meaningful part of the normalized key takes less than the given number of bytes, than it must be padded. Padding is typically required for variable length data types, such as strings. The padding uses a special character, either 0 or 0xff, depending on whether shorter values are sorted to the beginning or the end.

This method is similar to NormalizableKey.copyNormalizedKey(MemorySegment, int, int). In the case that multiple fields of a record contribute to the normalized key, it is crucial that the fields align on the byte field, i.e. that every field always takes up the exact same number of bytes.

Specified by:

putNormalizedKey in class TypeComparator<T>

Parameters:

record - The record for which to create the normalized key.

target - The byte array into which to write the normalized key bytes.

offset - The offset in the byte array, where to start writing the normalized key bytes.

numBytes - The number of bytes to be written exactly.

See Also:

NormalizableKey.copyNormalizedKey(MemorySegment, int, int)

writeWithKeyNormalization

public void writeWithKeyNormalization(T record,
                             DataOutputView target)
                               throws IOException

Description copied from class: TypeComparator

Writes the record in such a fashion that all keys are normalizing and at the beginning of the serialized data. This must only be used when for all the key fields the full normalized key is used. The method #supportsSerializationWithKeyNormalization() allows to check that.

Specified by:

writeWithKeyNormalization in class TypeComparator<T>

Parameters:

record - The record object into which to read the record data.

target - The stream to which to write the data,

Throws:

IOException

See Also:

TypeComparator.supportsSerializationWithKeyNormalization(), TypeComparator.readWithKeyDenormalization(Object, DataInputView), NormalizableKey.copyNormalizedKey(MemorySegment, int, int)

readWithKeyDenormalization

public T readWithKeyDenormalization(T reuse,
                           DataInputView source)
                             throws IOException

Description copied from class: TypeComparator

Reads the record back while de-normalizing the key fields. This must only be used when for all the key fields the full normalized key is used, which is hinted by the #supportsSerializationWithKeyNormalization() method.

Specified by:

readWithKeyDenormalization in class TypeComparator<T>

Parameters:

reuse - The reuse object into which to read the record data.

source - The stream from which to read the data,

Throws:

IOException

See Also:

TypeComparator.supportsSerializationWithKeyNormalization(), TypeComparator.writeWithKeyNormalization(Object, DataOutputView), NormalizableKey.copyNormalizedKey(MemorySegment, int, int)

invertNormalizedKey

public boolean invertNormalizedKey()

Description copied from class: TypeComparator

Flag whether normalized key comparisons should be inverted key should be interpreted inverted, i.e. descending.

Specified by:

invertNormalizedKey in class TypeComparator<T>

Returns:

True, if all normalized key comparisons should invert the sign of the comparison result, false if the normalized key should be used as is.