org.apache.flink.table.dataformat

Class BinaryRow

java.lang.Object

org.apache.flink.table.dataformat.BinarySection

org.apache.flink.table.dataformat.BinaryRow

All Implemented Interfaces:

BaseRow, BinaryFormat, TypeGetterSetters

public final class BinaryRow
extends BinarySection
implements BaseRow

A special row which is backed by MemorySegment instead of Object. It can significantly reduce the serialization/deserialization of Java objects.

A Row has two part: Fixed-length part and variable-length part.

Fixed-length part contains 1 byte header and null bit set and field values. Null bit set is used for null tracking and is aligned to 8-byte word boundaries. `Field values` holds fixed-length primitive types and variable-length values which can be stored in 8 bytes inside. If it do not fit the variable-length field, then store the length and offset of variable-length part.

Fixed-length part will certainly fall into a MemorySegment, which will speed up the read and write of field. During the write phase, if the target memory segment has less space than fixed length part size, we will skip the space. So the number of fields in a single Row cannot exceed the capacity of a single MemorySegment, if there are too many fields, we suggest that user set a bigger pageSize of MemorySegment.

Variable-length part may fall into multiple MemorySegments.

BinaryRow are influenced by Apache Spark UnsafeRow in project tungsten. The difference is that BinaryRow is placed on a discontinuous memory, and the variable length type can also be placed on a fixed length area (If it's short enough).

Field Summary

Fields

Modifier and Type	Field and Description
static int	HEADER_SIZE_IN_BITS
static boolean	LITTLE_ENDIAN

Fields inherited from class org.apache.flink.table.dataformat.BinarySection

offset, segments, sizeInBytes

Fields inherited from interface org.apache.flink.table.dataformat.BinaryFormat

HIGHEST_FIRST_BIT, HIGHEST_SECOND_TO_EIGHTH_BIT, MAX_FIX_PART_DATA_SIZE

Constructor Summary

Constructors

Constructor and Description
BinaryRow(int arity)

Method Summary

Modifier and Type	Method and Description
boolean	anyNull() The bit is 1 when the field is null.
boolean	anyNull(int[] fields)
static int	calculateBitSetWidthInBytes(int arity)
static int	calculateFixPartSizeInBytes(int arity)
void	clear()
BinaryRow	copy()
BinaryRow	copy(BinaryRow reuse)
boolean	equalsWithoutHeader(BaseRow o)
int	getArity() Get the number of fields in the BaseRow.
BaseArray	getArray(int pos) Get array value, internal format is BaseArray.
byte[]	getBinary(int pos) Get binary value, internal format is byte[].
boolean	getBoolean(int pos) Get boolean value.
byte	getByte(int pos) Get byte value.
Decimal	getDecimal(int pos, int precision, int scale) Get decimal value, internal format is Decimal.
double	getDouble(int pos) Get double value.
int	getFixedLengthPartSize()
float	getFloat(int pos) Get float value.
<T> BinaryGeneric<T>	getGeneric(int pos) Get generic value, internal format is BinaryGeneric.
byte	getHeader() The header represents the type of this Row.
int	getInt(int pos) Get int value.
long	getLong(int pos) Get long value.
BaseMap	getMap(int pos) Get map value, internal format is BaseMap.
BaseRow	getRow(int pos, int numFields) Get row value, internal format is BaseRow.
short	getShort(int pos) Get short value.
BinaryString	getString(int pos) Get string value, internal format is BinaryString.
int	hashCode()
static boolean	isInFixedLengthPart(org.apache.flink.table.types.logical.LogicalType type) If it is a fixed-length field, we can call this BinaryRow's setXX method for in-place updates.
static boolean	isMutable(org.apache.flink.table.types.logical.LogicalType type)
boolean	isNullAt(int pos) Because the specific row implementation such as BinaryRow uses the binary format.
void	setBoolean(int pos, boolean value) Set boolean value.
void	setByte(int pos, byte value) Set byte value.
void	setDecimal(int pos, Decimal value, int precision) Set the decimal column value.
void	setDouble(int pos, double value) Set double value.
void	setFloat(int pos, float value) Set float value.
void	setHeader(byte header) Set the byte header.
void	setInt(int pos, int value) Set int value.
void	setLong(int pos, long value) Set long value.
void	setNullAt(int i) Set null to this field.
void	setShort(int pos, short value) Set short value.
void	setTotalSize(int sizeInBytes)
static String	toOriginString(BaseRow row, org.apache.flink.table.types.logical.LogicalType[] types)
String	toOriginString(org.apache.flink.table.types.logical.LogicalType... types)

Methods inherited from class org.apache.flink.table.dataformat.BinarySection

equals, getOffset, getSegments, getSizeInBytes, pointTo, pointTo

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.table.dataformat.TypeGetterSetters

get

Methods inherited from interface org.apache.flink.table.dataformat.BinaryFormat

readBinaryFieldFromSegments, readBinaryStringFieldFromSegments

Field Detail

LITTLE_ENDIAN

public static final boolean LITTLE_ENDIAN

HEADER_SIZE_IN_BITS

public static final int HEADER_SIZE_IN_BITS

See Also:

Constant Field Values

Constructor Detail

BinaryRow

public BinaryRow(int arity)

Method Detail

calculateBitSetWidthInBytes

public static int calculateBitSetWidthInBytes(int arity)

calculateFixPartSizeInBytes

public static int calculateFixPartSizeInBytes(int arity)

isInFixedLengthPart

public static boolean isInFixedLengthPart(org.apache.flink.table.types.logical.LogicalType type)

If it is a fixed-length field, we can call this BinaryRow's setXX method for in-place updates. If it is variable-length field, can't use this method, because the underlying data is stored continuously.

isMutable

public static boolean isMutable(org.apache.flink.table.types.logical.LogicalType type)

getFixedLengthPartSize

public int getFixedLengthPartSize()

getArity

public int getArity()

Description copied from interface: BaseRow

Get the number of fields in the BaseRow.

Specified by:

getArity in interface BaseRow

Returns:

The number of fields in the BaseRow.

getHeader

public byte getHeader()

Description copied from interface: BaseRow

The header represents the type of this Row. Now just used in streaming. Now there are two message: ACCUMULATE_MSG and RETRACT_MSG.

Specified by:

getHeader in interface BaseRow

setHeader

public void setHeader(byte header)

Description copied from interface: BaseRow

Set the byte header.

Specified by:

setHeader in interface BaseRow

setTotalSize

public void setTotalSize(int sizeInBytes)

isNullAt

public boolean isNullAt(int pos)

Description copied from interface: TypeGetterSetters

Because the specific row implementation such as BinaryRow uses the binary format. We must first determine if it is null, and then make a specific get.

Specified by:

isNullAt in interface TypeGetterSetters

Returns:

true if this field is null.

setNullAt

public void setNullAt(int i)

Description copied from interface: TypeGetterSetters

Set null to this field.

Specified by:

setNullAt in interface TypeGetterSetters

setInt

public void setInt(int pos,
                   int value)

Description copied from interface: TypeGetterSetters

Set int value.

Specified by:

setInt in interface TypeGetterSetters

setLong

public void setLong(int pos,
                    long value)

Description copied from interface: TypeGetterSetters

Set long value.

Specified by:

setLong in interface TypeGetterSetters

setDouble

public void setDouble(int pos,
                      double value)

Description copied from interface: TypeGetterSetters

Set double value.

Specified by:

setDouble in interface TypeGetterSetters

setDecimal

public void setDecimal(int pos,
                       Decimal value,
                       int precision)

Description copied from interface: TypeGetterSetters

Set the decimal column value.

Note: Precision is compact: can call setNullAt when decimal is null. Precision is not compact: can not call setNullAt when decimal is null, must call setDecimal(i, null, precision) because we need update var-length-part.

Specified by:

setDecimal in interface TypeGetterSetters

setBoolean

public void setBoolean(int pos,
                       boolean value)

Description copied from interface: TypeGetterSetters

Set boolean value.

Specified by:

setBoolean in interface TypeGetterSetters

setShort

public void setShort(int pos,
                     short value)

Description copied from interface: TypeGetterSetters

Set short value.

Specified by:

setShort in interface TypeGetterSetters

setByte

public void setByte(int pos,
                    byte value)

Description copied from interface: TypeGetterSetters

Set byte value.

Specified by:

setByte in interface TypeGetterSetters

setFloat

public void setFloat(int pos,
                     float value)

Description copied from interface: TypeGetterSetters

Set float value.

Specified by:

setFloat in interface TypeGetterSetters

getBoolean

public boolean getBoolean(int pos)

Description copied from interface: TypeGetterSetters

Get boolean value.

Specified by:

getBoolean in interface TypeGetterSetters

getByte

public byte getByte(int pos)

Description copied from interface: TypeGetterSetters

Get byte value.

Specified by:

getByte in interface TypeGetterSetters

getShort

public short getShort(int pos)

Description copied from interface: TypeGetterSetters

Get short value.

Specified by:

getShort in interface TypeGetterSetters

getInt

public int getInt(int pos)

Description copied from interface: TypeGetterSetters

Get int value.

Specified by:

getInt in interface TypeGetterSetters

getLong

public long getLong(int pos)

Description copied from interface: TypeGetterSetters

Get long value.

Specified by:

getLong in interface TypeGetterSetters

getFloat

public float getFloat(int pos)

Description copied from interface: TypeGetterSetters

Get float value.

Specified by:

getFloat in interface TypeGetterSetters

getDouble

public double getDouble(int pos)

Description copied from interface: TypeGetterSetters

Get double value.

Specified by:

getDouble in interface TypeGetterSetters

getString

public BinaryString getString(int pos)

Description copied from interface: TypeGetterSetters

Get string value, internal format is BinaryString.

Specified by:

getString in interface TypeGetterSetters

getDecimal

public Decimal getDecimal(int pos,
                          int precision,
                          int scale)

Description copied from interface: TypeGetterSetters

Get decimal value, internal format is Decimal.

Specified by:

getDecimal in interface TypeGetterSetters

getGeneric

public <T> BinaryGeneric<T> getGeneric(int pos)

Description copied from interface: TypeGetterSetters

Get generic value, internal format is BinaryGeneric.

Specified by:

getGeneric in interface TypeGetterSetters

getBinary

public byte[] getBinary(int pos)

Description copied from interface: TypeGetterSetters

Get binary value, internal format is byte[].

Specified by:

getBinary in interface TypeGetterSetters

getArray

public BaseArray getArray(int pos)

Description copied from interface: TypeGetterSetters

Get array value, internal format is BaseArray.

Specified by:

getArray in interface TypeGetterSetters

getMap

public BaseMap getMap(int pos)

Description copied from interface: TypeGetterSetters

Get map value, internal format is BaseMap.

Specified by:

getMap in interface TypeGetterSetters

getRow

public BaseRow getRow(int pos,
                      int numFields)

Description copied from interface: TypeGetterSetters

Get row value, internal format is BaseRow.

Specified by:

getRow in interface TypeGetterSetters

anyNull

public boolean anyNull()

The bit is 1 when the field is null. Default is 0.

anyNull

public boolean anyNull(int[] fields)

copy

public BinaryRow copy()

copy

public BinaryRow copy(BinaryRow reuse)

clear

public void clear()

hashCode

public int hashCode()

Overrides:

hashCode in class BinarySection

toOriginString

public String toOriginString(org.apache.flink.table.types.logical.LogicalType... types)

toOriginString

public static String toOriginString(BaseRow row,
                                    org.apache.flink.table.types.logical.LogicalType[] types)

equalsWithoutHeader

public boolean equalsWithoutHeader(BaseRow o)