org.apache.hudi.common.model

Class HoodieAvroRecord<T extends HoodieRecordPayload>

java.lang.Object

org.apache.hudi.common.model.HoodieRecord<T>

org.apache.hudi.common.model.HoodieAvroRecord<T>

Type Parameters:

T - payload implementation.

All Implemented Interfaces:

com.esotericsoftware.kryo.KryoSerializable, Serializable, HoodieRecordCompatibilityInterface

public class HoodieAvroRecord<T extends HoodieRecordPayload>
extends HoodieRecord<T>

Implementation of HoodieRecord using Avro payload.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hudi.common.model.HoodieRecord

HoodieRecord.HoodieMetadataField, HoodieRecord.HoodieRecordType

Field Summary

Fields inherited from class org.apache.hudi.common.model.HoodieRecord

COMMIT_SEQNO_METADATA_FIELD, COMMIT_SEQNO_METADATA_FIELD_ORD, COMMIT_TIME_METADATA_FIELD, COMMIT_TIME_METADATA_FIELD_ORD, currentLocation, data, FILENAME_META_FIELD_ORD, FILENAME_METADATA_FIELD, HOODIE_IS_DELETED_FIELD, HOODIE_META_COLUMNS, HOODIE_META_COLUMNS_NAME_TO_POS, HOODIE_META_COLUMNS_WITH_OPERATION, ignoreIndexUpdate, key, metaData, newLocation, operation, OPERATION_METADATA_FIELD, PARTITION_PATH_META_FIELD_ORD, PARTITION_PATH_METADATA_FIELD, RECORD_KEY_META_FIELD_ORD, RECORD_KEY_METADATA_FIELD, SENTINEL

Constructor Summary

Constructors

Constructor and Description
HoodieAvroRecord()
HoodieAvroRecord(HoodieKey key, T data)
HoodieAvroRecord(HoodieKey key, T data, HoodieOperation operation)
HoodieAvroRecord(HoodieKey key, T data, HoodieOperation operation, HoodieRecordLocation currentLocation, HoodieRecordLocation newLocation)
HoodieAvroRecord(HoodieRecord<T> record)

Method Summary

Modifier and Type	Method and Description
HoodieRecord<T>	copy() This is used to copy data.
Object[]	getColumnValues(org.apache.avro.Schema recordSchema, String[] columns, boolean consistentLogicalTimestampEnabled) Get column in record to support RDDCustomColumnsSortPartitioner
T	getData()
Option<Map<String,String>>	getMetadata()
Comparable<?>	getOrderingValue(org.apache.avro.Schema recordSchema, Properties props)
String	getRecordKey(org.apache.avro.Schema recordSchema, Option<BaseKeyGenerator> keyGeneratorOpt)
String	getRecordKey(org.apache.avro.Schema recordSchema, String keyFieldName)
HoodieRecord.HoodieRecordType	getRecordType()
boolean	isDelete(org.apache.avro.Schema recordSchema, Properties props)
HoodieRecord	joinWith(HoodieRecord other, org.apache.avro.Schema targetSchema) Support bootstrap.
HoodieRecord<T>	newInstance()
HoodieRecord<T>	newInstance(HoodieKey key)
HoodieRecord<T>	newInstance(HoodieKey key, HoodieOperation op)
HoodieRecord	prependMetaFields(org.apache.avro.Schema recordSchema, org.apache.avro.Schema targetSchema, MetadataValues metadataValues, Properties props) Rewrites record into new target schema containing Hudi-specific meta-fields NOTE: This operation is idempotent
protected T	readRecordPayload(com.esotericsoftware.kryo.Kryo kryo, com.esotericsoftware.kryo.io.Input input)
HoodieRecord	rewriteRecordWithNewSchema(org.apache.avro.Schema recordSchema, Properties props, org.apache.avro.Schema newSchema, Map<String,String> renameCols) Support schema evolution.
boolean	shouldIgnore(org.apache.avro.Schema recordSchema, Properties props) Is EmptyRecord.
Option<HoodieAvroIndexedRecord>	toIndexedRecord(org.apache.avro.Schema recordSchema, Properties props)
HoodieRecord	truncateRecordKey(org.apache.avro.Schema recordSchema, Properties props, String keyFieldName) This method used to overwrite record key field.
HoodieRecord	wrapIntoHoodieRecordPayloadWithKeyGen(org.apache.avro.Schema recordSchema, Properties props, Option<BaseKeyGenerator> keyGen) This method used to extract HoodieKey through keyGenerator.
HoodieRecord	wrapIntoHoodieRecordPayloadWithParams(org.apache.avro.Schema recordSchema, Properties props, Option<Pair<String,String>> simpleKeyGenFieldsOpt, Boolean withOperation, Option<String> partitionNameOp, Boolean populateMetaFields, Option<org.apache.avro.Schema> schemaWithoutMetaFields) This method used to extract HoodieKey not through keyGenerator.
protected void	writeRecordPayload(T payload, com.esotericsoftware.kryo.Kryo kryo, com.esotericsoftware.kryo.io.Output output)

Methods inherited from class org.apache.hudi.common.model.HoodieRecord

checkState, clearNewLocation, deflate, equals, generateSequenceId, getCurrentLocation, getCurrentPosition, getIgnoreIndexUpdate, getKey, getNewLocation, getOperation, getPartitionPath, getRecordKey, hashCode, hasMetaFields, isCurrentLocationKnown, read, rewriteRecordWithNewSchema, seal, setCurrentLocation, setIgnoreIndexUpdate, setNewLocation, toString, unseal, write

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

HoodieAvroRecord

public HoodieAvroRecord(HoodieKey key,
                        T data)

HoodieAvroRecord

public HoodieAvroRecord(HoodieKey key,
                        T data,
                        HoodieOperation operation)

HoodieAvroRecord

public HoodieAvroRecord(HoodieRecord<T> record)

HoodieAvroRecord

public HoodieAvroRecord(HoodieKey key,
                        T data,
                        HoodieOperation operation,
                        HoodieRecordLocation currentLocation,
                        HoodieRecordLocation newLocation)

HoodieAvroRecord

public HoodieAvroRecord()

Method Detail

newInstance

public HoodieRecord<T> newInstance()

Specified by:

newInstance in class HoodieRecord<T extends HoodieRecordPayload>

newInstance

public HoodieRecord<T> newInstance(HoodieKey key,
                                   HoodieOperation op)

Specified by:

newInstance in class HoodieRecord<T extends HoodieRecordPayload>

newInstance

public HoodieRecord<T> newInstance(HoodieKey key)

Specified by:

newInstance in class HoodieRecord<T extends HoodieRecordPayload>

getData

public T getData()

Overrides:

getData in class HoodieRecord<T extends HoodieRecordPayload>

getOrderingValue

public Comparable<?> getOrderingValue(org.apache.avro.Schema recordSchema,
                                      Properties props)

Specified by:

getOrderingValue in class HoodieRecord<T extends HoodieRecordPayload>

getRecordKey

public String getRecordKey(org.apache.avro.Schema recordSchema,
                           Option<BaseKeyGenerator> keyGeneratorOpt)

Specified by:

getRecordKey in class HoodieRecord<T extends HoodieRecordPayload>

getRecordKey

public String getRecordKey(org.apache.avro.Schema recordSchema,
                           String keyFieldName)

Specified by:

getRecordKey in class HoodieRecord<T extends HoodieRecordPayload>

getRecordType

public HoodieRecord.HoodieRecordType getRecordType()

Specified by:

getRecordType in class HoodieRecord<T extends HoodieRecordPayload>

getColumnValues

public Object[] getColumnValues(org.apache.avro.Schema recordSchema,
                                String[] columns,
                                boolean consistentLogicalTimestampEnabled)

Description copied from class: HoodieRecord

Get column in record to support RDDCustomColumnsSortPartitioner

Specified by:

getColumnValues in class HoodieRecord<T extends HoodieRecordPayload>

Returns:

joinWith

public HoodieRecord joinWith(HoodieRecord other,
                             org.apache.avro.Schema targetSchema)

Description copied from class: HoodieRecord

Support bootstrap.

Specified by:

joinWith in class HoodieRecord<T extends HoodieRecordPayload>

prependMetaFields

public HoodieRecord prependMetaFields(org.apache.avro.Schema recordSchema,
                                      org.apache.avro.Schema targetSchema,
                                      MetadataValues metadataValues,
                                      Properties props)

Description copied from class: HoodieRecord

Rewrites record into new target schema containing Hudi-specific meta-fields NOTE: This operation is idempotent

Specified by:

prependMetaFields in class HoodieRecord<T extends HoodieRecordPayload>

rewriteRecordWithNewSchema

public HoodieRecord rewriteRecordWithNewSchema(org.apache.avro.Schema recordSchema,
                                               Properties props,
                                               org.apache.avro.Schema newSchema,
                                               Map<String,String> renameCols)

Description copied from class: HoodieRecord

Support schema evolution.

Specified by:

rewriteRecordWithNewSchema in class HoodieRecord<T extends HoodieRecordPayload>

truncateRecordKey

public HoodieRecord truncateRecordKey(org.apache.avro.Schema recordSchema,
                                      Properties props,
                                      String keyFieldName)
                               throws IOException

Description copied from interface: HoodieRecordCompatibilityInterface

This method used to overwrite record key field.

Throws:

IOException

isDelete

public boolean isDelete(org.apache.avro.Schema recordSchema,
                        Properties props)
                 throws IOException

Specified by:

isDelete in class HoodieRecord<T extends HoodieRecordPayload>

Throws:

IOException

shouldIgnore

public boolean shouldIgnore(org.apache.avro.Schema recordSchema,
                            Properties props)
                     throws IOException

Description copied from class: HoodieRecord

Is EmptyRecord. Generated by ExpressionPayload.

Specified by:

shouldIgnore in class HoodieRecord<T extends HoodieRecordPayload>

Throws:

IOException

copy

public HoodieRecord<T> copy()

Description copied from class: HoodieRecord

This is used to copy data.

Specified by:

copy in class HoodieRecord<T extends HoodieRecordPayload>

wrapIntoHoodieRecordPayloadWithParams

public HoodieRecord wrapIntoHoodieRecordPayloadWithParams(org.apache.avro.Schema recordSchema,
                                                          Properties props,
                                                          Option<Pair<String,String>> simpleKeyGenFieldsOpt,
                                                          Boolean withOperation,
                                                          Option<String> partitionNameOp,
                                                          Boolean populateMetaFields,
                                                          Option<org.apache.avro.Schema> schemaWithoutMetaFields)
                                                   throws IOException

Description copied from interface: HoodieRecordCompatibilityInterface

This method used to extract HoodieKey not through keyGenerator.

Throws:

IOException

wrapIntoHoodieRecordPayloadWithKeyGen

public HoodieRecord wrapIntoHoodieRecordPayloadWithKeyGen(org.apache.avro.Schema recordSchema,
                                                          Properties props,
                                                          Option<BaseKeyGenerator> keyGen)

Description copied from interface: HoodieRecordCompatibilityInterface

This method used to extract HoodieKey through keyGenerator. This method used in ClusteringExecutionStrategy.

getMetadata

public Option<Map<String,String>> getMetadata()

Specified by:

getMetadata in class HoodieRecord<T extends HoodieRecordPayload>

toIndexedRecord

public Option<HoodieAvroIndexedRecord> toIndexedRecord(org.apache.avro.Schema recordSchema,
                                                       Properties props)
                                                throws IOException

Throws:

IOException

writeRecordPayload

protected final void writeRecordPayload(T payload,
                                        com.esotericsoftware.kryo.Kryo kryo,
                                        com.esotericsoftware.kryo.io.Output output)

Specified by:

writeRecordPayload in class HoodieRecord<T extends HoodieRecordPayload>

readRecordPayload

protected final T readRecordPayload(com.esotericsoftware.kryo.Kryo kryo,
                                    com.esotericsoftware.kryo.io.Input input)

Specified by:

readRecordPayload in class HoodieRecord<T extends HoodieRecordPayload>