org.apache.hudi.common.table.log

Class HoodieMergedLogRecordReader<T>

java.lang.Object

org.apache.hudi.common.table.log.BaseHoodieLogRecordReader<T>

org.apache.hudi.common.table.log.HoodieMergedLogRecordReader<T>

Type Parameters:

T - type of engine-specific record representation.

All Implemented Interfaces:

Closeable, AutoCloseable, Iterable<Pair<Option<T>,Map<String,Object>>>

public class HoodieMergedLogRecordReader<T>
extends BaseHoodieLogRecordReader<T>
implements Iterable<Pair<Option<T>,Map<String,Object>>>, Closeable

A log record reader that merges the records with the same record key.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	HoodieMergedLogRecordReader.Builder<T> Builder used to build HoodieUnMergedLogRecordScanner.

Field Summary

Fields

Modifier and Type	Field and Description
HoodieTimer	timer

Fields inherited from class org.apache.hudi.common.table.log.BaseHoodieLogRecordReader

forceFullScan, hoodieTableMetaClient, logFilePaths, preCombineField, readerContext, readerSchema, recordBuffer, recordMergeMode, recordMerger, recordType

Method Summary

Modifier and Type	Method and Description
void	close()
long	getNumMergedRecordsInLog()
Map<Serializable,Pair<Option<T>,Map<String,Object>>>	getRecords()
HoodieRecord.HoodieRecordType	getRecordType()
long	getTotalTimeTakenToReadAndMergeBlocks()
Iterator<Pair<Option<T>,Map<String,Object>>>	iterator()
static HoodieMergedLogRecordReader.Builder	newBuilder() Returns the builder for HoodieMergedLogRecordReader.
void	scan() Scans delta-log files processing blocks
void	scan(boolean skipProcessingBlocks)
void	scanByFullKeys(List<String> keys) Provides incremental scanning capability where only provided keys will be looked up in the delta-log files, scanned and subsequently materialized into the internal cache
void	scanByKeyPrefixes(List<String> keyPrefixes) Provides incremental scanning capability where only keys matching provided key-prefixes will be looked up in the delta-log files, scanned and subsequently materialized into the internal cache

Methods inherited from class org.apache.hudi.common.table.log.BaseHoodieLogRecordReader

getCurrentInstantLogBlocks, getPartitionNameOverride, getPayloadClassFQN, getPayloadProps, getProgress, getTotalCorruptBlocks, getTotalLogBlocks, getTotalLogFiles, getTotalLogRecords, getTotalRollbacks, getValidBlockInstants, isWithOperationField, scanInternal

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.lang.Iterable

forEach, spliterator

Field Detail

timer

public final HoodieTimer timer

Method Detail

scan

public final void scan()

Scans delta-log files processing blocks

scan

public final void scan(boolean skipProcessingBlocks)

scanByFullKeys

public void scanByFullKeys(List<String> keys)

Provides incremental scanning capability where only provided keys will be looked up in the delta-log files, scanned and subsequently materialized into the internal cache

Parameters:

keys - to be looked up

scanByKeyPrefixes

public void scanByKeyPrefixes(List<String> keyPrefixes)

Provides incremental scanning capability where only keys matching provided key-prefixes will be looked up in the delta-log files, scanned and subsequently materialized into the internal cache

Parameters:

keyPrefixes - to be looked up

iterator

public Iterator<Pair<Option<T>,Map<String,Object>>> iterator()

Specified by:

iterator in interface Iterable<Pair<Option<T>,Map<String,Object>>>

getRecords

public Map<Serializable,Pair<Option<T>,Map<String,Object>>> getRecords()

getRecordType

public HoodieRecord.HoodieRecordType getRecordType()

getNumMergedRecordsInLog

public long getNumMergedRecordsInLog()

newBuilder

public static HoodieMergedLogRecordReader.Builder newBuilder()

Returns the builder for HoodieMergedLogRecordReader.

getTotalTimeTakenToReadAndMergeBlocks

public long getTotalTimeTakenToReadAndMergeBlocks()

close

public void close()

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable