org.apache.hudi.common.model

Class EventTimeAvroPayload

java.lang.Object

org.apache.hudi.common.model.BaseAvroPayload

org.apache.hudi.common.model.OverwriteWithLatestAvroPayload

org.apache.hudi.common.model.DefaultHoodieRecordPayload

org.apache.hudi.common.model.EventTimeAvroPayload

All Implemented Interfaces:

Serializable, HoodieRecordPayload<OverwriteWithLatestAvroPayload>

public class EventTimeAvroPayload
extends DefaultHoodieRecordPayload

The only difference with DefaultHoodieRecordPayload is that is does not track the event time metadata for efficiency.

See Also:

Serialized Form

Field Summary

Fields inherited from class org.apache.hudi.common.model.DefaultHoodieRecordPayload

DELETE_KEY, DELETE_MARKER, METADATA_EVENT_TIME_KEY

Fields inherited from class org.apache.hudi.common.model.BaseAvroPayload

isDeletedRecord, orderingVal, recordBytes

Constructor Summary

Constructors

Constructor and Description
EventTimeAvroPayload(org.apache.avro.generic.GenericRecord record, Comparable orderingVal)
EventTimeAvroPayload(Option<org.apache.avro.generic.GenericRecord> record)

Method Summary

Modifier and Type	Method and Description
Option<org.apache.avro.generic.IndexedRecord>	combineAndGetUpdateValue(org.apache.avro.generic.IndexedRecord currentValue, org.apache.avro.Schema schema, Properties properties) This methods lets you write custom merging/combining logic to produce new values as a function of current value on storage and whats contained in this object.
Option<org.apache.avro.generic.IndexedRecord>	getInsertValue(org.apache.avro.Schema schema, Properties properties) Generates an avro record out of the given HoodieRecordPayload, to be written out to storage.
Option<Map<String,String>>	getMetadata() This method can be used to extract some metadata from HoodieRecordPayload.

Methods inherited from class org.apache.hudi.common.model.DefaultHoodieRecordPayload

isDeleted, isDeleteRecord, needUpdatingPersistedRecord

Methods inherited from class org.apache.hudi.common.model.OverwriteWithLatestAvroPayload

combineAndGetUpdateValue, getInsertValue, getOrderingValue, overwriteField, preCombine

Methods inherited from class org.apache.hudi.common.model.BaseAvroPayload

canProduceSentinel, getOrderingVal, isDeleteRecord

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hudi.common.model.HoodieRecordPayload

preCombine, preCombine

Constructor Detail

EventTimeAvroPayload

public EventTimeAvroPayload(org.apache.avro.generic.GenericRecord record,
                            Comparable orderingVal)

EventTimeAvroPayload

public EventTimeAvroPayload(Option<org.apache.avro.generic.GenericRecord> record)

Method Detail

combineAndGetUpdateValue

public Option<org.apache.avro.generic.IndexedRecord> combineAndGetUpdateValue(org.apache.avro.generic.IndexedRecord currentValue,
                                                                              org.apache.avro.Schema schema,
                                                                              Properties properties)
                                                                       throws IOException

Description copied from interface: HoodieRecordPayload

This methods lets you write custom merging/combining logic to produce new values as a function of current value on storage and whats contained in this object. Implementations can leverage properties if required.

eg: 1) You are updating counters, you may want to add counts to currentValue and write back updated counts 2) You may be reading DB redo logs, and merge them with current image for a database row on storage

Specified by:

combineAndGetUpdateValue in interface HoodieRecordPayload<OverwriteWithLatestAvroPayload>

Overrides:

combineAndGetUpdateValue in class DefaultHoodieRecordPayload

Parameters:

currentValue - Current value in storage, to merge/combine this payload with

schema - Schema used for record

properties - Payload related properties. For example pass the ordering field(s) name to extract from value in storage.

Returns:

new combined/merged value to be written back to storage. EMPTY to skip writing this record.

Throws:

IOException

getInsertValue

public Option<org.apache.avro.generic.IndexedRecord> getInsertValue(org.apache.avro.Schema schema,
                                                                    Properties properties)
                                                             throws IOException

Description copied from interface: HoodieRecordPayload

Generates an avro record out of the given HoodieRecordPayload, to be written out to storage. Called when writing a new value for the given HoodieKey, wherein there is no existing record in storage to be combined against. (i.e insert) Return EMPTY to skip writing this record. Implementations can leverage properties if required.

Specified by:

getInsertValue in interface HoodieRecordPayload<OverwriteWithLatestAvroPayload>

Overrides:

getInsertValue in class DefaultHoodieRecordPayload

Parameters:

schema - Schema used for record

properties - Payload related properties. For example pass the ordering field(s) name to extract from value in storage.

Returns:

the IndexedRecord to be inserted.

Throws:

IOException

getMetadata

public Option<Map<String,String>> getMetadata()

Description copied from interface: HoodieRecordPayload

This method can be used to extract some metadata from HoodieRecordPayload. The metadata is passed to WriteStatus.markSuccess() and WriteStatus.markFailure() in order to compute some aggregate metrics using the metadata in the context of a write success or failure.

Specified by:

getMetadata in interface HoodieRecordPayload<OverwriteWithLatestAvroPayload>

Overrides:

getMetadata in class DefaultHoodieRecordPayload

Returns:

the metadata in the form of Map if any.