org.apache.hudi.table.format

Class FormatUtils

java.lang.Object

org.apache.hudi.table.format.FormatUtils

public class FormatUtils
extends Object

Utilities for format.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	FormatUtils.BoundedMemoryRecords Utility to read and buffer the records in the unMerged log record scanner.

Method Summary

Modifier and Type	Method and Description
static org.apache.avro.generic.GenericRecord	buildAvroRecordBySchema(org.apache.avro.generic.IndexedRecord record, org.apache.avro.Schema requiredSchema, int[] requiredPos, org.apache.avro.generic.GenericRecordBuilder recordBuilder)
static org.apache.flink.types.RowKind	getRowKindSafely(org.apache.avro.generic.IndexedRecord record, int index) Returns the RowKind of the given record, never null.
static HoodieMergedLogRecordScanner	logScanner(List<String> logPaths, org.apache.avro.Schema logSchema, String latestInstantTime, HoodieWriteConfig writeConfig, org.apache.hadoop.conf.Configuration hadoopConf)
static HoodieMergedLogRecordScanner	logScanner(MergeOnReadInputSplit split, org.apache.avro.Schema logSchema, InternalSchema internalSchema, org.apache.flink.configuration.Configuration flinkConf, org.apache.hadoop.conf.Configuration hadoopConf)
static void	setRowKind(org.apache.flink.table.data.RowData rowData, org.apache.avro.generic.IndexedRecord record, int index) Sets up the row kind to the row data rowData from the resolved operation.
static ExternalSpillableMap<String,byte[]>	spillableMap(HoodieWriteConfig writeConfig, long maxCompactionMemoryInBytes)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

setRowKind

public static void setRowKind(org.apache.flink.table.data.RowData rowData,
                              org.apache.avro.generic.IndexedRecord record,
                              int index)

Sets up the row kind to the row data rowData from the resolved operation.

getRowKindSafely

public static org.apache.flink.types.RowKind getRowKindSafely(org.apache.avro.generic.IndexedRecord record,
                                                              int index)

Returns the RowKind of the given record, never null. Returns RowKind.INSERT when the given field value not found.

buildAvroRecordBySchema

public static org.apache.avro.generic.GenericRecord buildAvroRecordBySchema(org.apache.avro.generic.IndexedRecord record,
                                                                            org.apache.avro.Schema requiredSchema,
                                                                            int[] requiredPos,
                                                                            org.apache.avro.generic.GenericRecordBuilder recordBuilder)

spillableMap

public static ExternalSpillableMap<String,byte[]> spillableMap(HoodieWriteConfig writeConfig,
                                                               long maxCompactionMemoryInBytes)

logScanner

public static HoodieMergedLogRecordScanner logScanner(MergeOnReadInputSplit split,
                                                      org.apache.avro.Schema logSchema,
                                                      InternalSchema internalSchema,
                                                      org.apache.flink.configuration.Configuration flinkConf,
                                                      org.apache.hadoop.conf.Configuration hadoopConf)

logScanner

public static HoodieMergedLogRecordScanner logScanner(List<String> logPaths,
                                                      org.apache.avro.Schema logSchema,
                                                      String latestInstantTime,
                                                      HoodieWriteConfig writeConfig,
                                                      org.apache.hadoop.conf.Configuration hadoopConf)