org.apache.hudi.common.table.read

Class HoodieUnmergedFileGroupRecordBuffer<T>

java.lang.Object

org.apache.hudi.common.table.read.HoodieBaseFileGroupRecordBuffer<T>

org.apache.hudi.common.table.read.HoodieUnmergedFileGroupRecordBuffer<T>

All Implemented Interfaces:

HoodieFileGroupRecordBuffer<T>

public class HoodieUnmergedFileGroupRecordBuffer<T>
extends HoodieBaseFileGroupRecordBuffer<T>

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.hudi.common.table.read.HoodieFileGroupRecordBuffer

HoodieFileGroupRecordBuffer.BufferType

Field Summary

Fields inherited from class org.apache.hudi.common.table.read.HoodieBaseFileGroupRecordBuffer

baseFileIterator, enablePartialMerging, hoodieTableMetaClient, internalSchema, logRecordIterator, nextRecord, partitionNameOverrideOpt, partitionPathFieldOpt, props, readerContext, readerSchema, recordMergeMode, recordMerger, records

Constructor Summary

Constructors

Constructor and Description
HoodieUnmergedFileGroupRecordBuffer(HoodieReaderContext<T> readerContext, HoodieTableMetaClient hoodieTableMetaClient, Option<String> partitionNameOverrideOpt, Option<String[]> partitionPathFieldOpt, HoodieRecordMerger recordMerger, TypedProperties props)

Method Summary

Modifier and Type	Method and Description
boolean	containsLogRecord(String recordKey) Check if a record exists in the buffered records.
protected boolean	doHasNext() This allows hasNext() to be called multiple times without incrementing the iterator by more than 1 record.
HoodieFileGroupRecordBuffer.BufferType	getBufferType()
Iterator<Pair<Option<T>,Map<String,Object>>>	getLogRecordIterator()
void	processDataBlock(HoodieDataBlock dataBlock, Option<KeySpec> keySpecOpt) Process a log data block, and store the resulting records into the buffer.
void	processDeleteBlock(HoodieDeleteBlock deleteBlock) Process a log delete block, and store the resulting records into the buffer.
void	processNextDataRecord(T record, Map<String,Object> metadata, Serializable index) Process a next record in a log data block.
void	processNextDeletedRecord(DeleteRecord deleteRecord, Serializable index) Process next delete record.

Methods inherited from class org.apache.hudi.common.table.read.HoodieBaseFileGroupRecordBuffer

close, composeEvolvedSchemaTransformer, doProcessNextDataRecord, doProcessNextDeletedRecord, getLogRecords, getRecordsIterator, getSchemaTransformerWithEvolvedSchema, hasNext, hasNextBaseRecord, hasNextLogRecord, merge, next, setBaseFileIterator, size

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HoodieUnmergedFileGroupRecordBuffer

public HoodieUnmergedFileGroupRecordBuffer(HoodieReaderContext<T> readerContext,
                                           HoodieTableMetaClient hoodieTableMetaClient,
                                           Option<String> partitionNameOverrideOpt,
                                           Option<String[]> partitionPathFieldOpt,
                                           HoodieRecordMerger recordMerger,
                                           TypedProperties props)

Method Detail

doHasNext

protected boolean doHasNext()
                     throws IOException

Description copied from class: HoodieBaseFileGroupRecordBuffer

This allows hasNext() to be called multiple times without incrementing the iterator by more than 1 record. It does come with the caveat that hasNext() must be called every time before next(). But that is pretty expected behavior and every user should be doing that anyway.

Specified by:

doHasNext in class HoodieBaseFileGroupRecordBuffer<T>

Throws:

IOException

getLogRecordIterator

public Iterator<Pair<Option<T>,Map<String,Object>>> getLogRecordIterator()

Specified by:

getLogRecordIterator in interface HoodieFileGroupRecordBuffer<T>

Overrides:

getLogRecordIterator in class HoodieBaseFileGroupRecordBuffer<T>

Returns:

An iterator on the log records.

getBufferType

public HoodieFileGroupRecordBuffer.BufferType getBufferType()

Returns:

The merge strategy implemented.

processDataBlock

public void processDataBlock(HoodieDataBlock dataBlock,
                             Option<KeySpec> keySpecOpt)

Description copied from interface: HoodieFileGroupRecordBuffer

Process a log data block, and store the resulting records into the buffer.

processNextDataRecord

public void processNextDataRecord(T record,
                                  Map<String,Object> metadata,
                                  Serializable index)

Description copied from interface: HoodieFileGroupRecordBuffer

Process a next record in a log data block.

processDeleteBlock

public void processDeleteBlock(HoodieDeleteBlock deleteBlock)

Description copied from interface: HoodieFileGroupRecordBuffer

Process a log delete block, and store the resulting records into the buffer.

processNextDeletedRecord

public void processNextDeletedRecord(DeleteRecord deleteRecord,
                                     Serializable index)

Description copied from interface: HoodieFileGroupRecordBuffer

Process next delete record.

containsLogRecord

public boolean containsLogRecord(String recordKey)

Description copied from interface: HoodieFileGroupRecordBuffer

Check if a record exists in the buffered records.