org.apache.hudi.common.model.debezium

Class PostgresDebeziumAvroPayload

java.lang.Object

org.apache.hudi.common.model.BaseAvroPayload

org.apache.hudi.common.model.OverwriteWithLatestAvroPayload

org.apache.hudi.common.model.debezium.AbstractDebeziumAvroPayload

org.apache.hudi.common.model.debezium.PostgresDebeziumAvroPayload

All Implemented Interfaces:

Serializable, HoodieRecordPayload<OverwriteWithLatestAvroPayload>

public class PostgresDebeziumAvroPayload
extends AbstractDebeziumAvroPayload

Provides support for seamlessly applying changes captured via Debezium for PostgresDB.

Debezium change event types are determined for the op field in the payload

- For inserts, op=i - For deletes, op=d - For updates, op=u - For snapshot inserts, op=r

This payload implementation will issue matching insert, delete, updates against the hudi table

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
static String	DEBEZIUM_TOASTED_VALUE

Fields inherited from class org.apache.hudi.common.model.BaseAvroPayload

isDeletedRecord, orderingVal, recordBytes

Constructor Summary

Constructors

Constructor and Description
PostgresDebeziumAvroPayload(org.apache.avro.generic.GenericRecord record, Comparable orderingVal)
PostgresDebeziumAvroPayload(Option<org.apache.avro.generic.GenericRecord> record)

Method Summary

Modifier and Type	Method and Description
Option<org.apache.avro.generic.IndexedRecord>	combineAndGetUpdateValue(org.apache.avro.generic.IndexedRecord currentValue, org.apache.avro.Schema schema) This methods is deprecated.
Option<org.apache.avro.generic.IndexedRecord>	combineAndGetUpdateValue(org.apache.avro.generic.IndexedRecord currentValue, org.apache.avro.Schema schema, Properties properties) This methods lets you write custom merging/combining logic to produce new values as a function of current value on storage and whats contained in this object.
protected boolean	shouldPickCurrentRecord(org.apache.avro.generic.IndexedRecord currentRecord, org.apache.avro.generic.IndexedRecord insertRecord, org.apache.avro.Schema schema)

Methods inherited from class org.apache.hudi.common.model.debezium.AbstractDebeziumAvroPayload

getInsertValue, isDeleteRecord

Methods inherited from class org.apache.hudi.common.model.OverwriteWithLatestAvroPayload

getOrderingValue, overwriteField, preCombine

Methods inherited from class org.apache.hudi.common.model.BaseAvroPayload

canProduceSentinel, getOrderingVal, isDeleted

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hudi.common.model.HoodieRecordPayload

getInsertValue, getMetadata, preCombine, preCombine

Field Detail

DEBEZIUM_TOASTED_VALUE

public static final String DEBEZIUM_TOASTED_VALUE

See Also:

Constant Field Values

Constructor Detail

PostgresDebeziumAvroPayload

public PostgresDebeziumAvroPayload(org.apache.avro.generic.GenericRecord record,
                                   Comparable orderingVal)

PostgresDebeziumAvroPayload

public PostgresDebeziumAvroPayload(Option<org.apache.avro.generic.GenericRecord> record)

Method Detail

shouldPickCurrentRecord

protected boolean shouldPickCurrentRecord(org.apache.avro.generic.IndexedRecord currentRecord,
                                          org.apache.avro.generic.IndexedRecord insertRecord,
                                          org.apache.avro.Schema schema)
                                   throws IOException

Specified by:

shouldPickCurrentRecord in class AbstractDebeziumAvroPayload

Throws:

IOException

combineAndGetUpdateValue

public Option<org.apache.avro.generic.IndexedRecord> combineAndGetUpdateValue(org.apache.avro.generic.IndexedRecord currentValue,
                                                                              org.apache.avro.Schema schema,
                                                                              Properties properties)
                                                                       throws IOException

Description copied from interface: HoodieRecordPayload

This methods lets you write custom merging/combining logic to produce new values as a function of current value on storage and whats contained in this object. Implementations can leverage properties if required.

eg: 1) You are updating counters, you may want to add counts to currentValue and write back updated counts 2) You may be reading DB redo logs, and merge them with current image for a database row on storage

Parameters:

currentValue - Current value in storage, to merge/combine this payload with

schema - Schema used for record

properties - Payload related properties. For example pass the ordering field(s) name to extract from value in storage.

Returns:

new combined/merged value to be written back to storage. EMPTY to skip writing this record.

Throws:

IOException

combineAndGetUpdateValue

public Option<org.apache.avro.generic.IndexedRecord> combineAndGetUpdateValue(org.apache.avro.generic.IndexedRecord currentValue,
                                                                              org.apache.avro.Schema schema)
                                                                       throws IOException

Description copied from interface: HoodieRecordPayload

This methods is deprecated. Please refer to HoodieRecordPayload.combineAndGetUpdateValue(IndexedRecord, Schema, Properties) for java docs.

Specified by:

combineAndGetUpdateValue in interface HoodieRecordPayload<OverwriteWithLatestAvroPayload>

Overrides:

combineAndGetUpdateValue in class AbstractDebeziumAvroPayload

Throws:

IOException