org.apache.hudi.io

Class HoodieSortedMergeHandle<T extends HoodieRecordPayload,I,K,O>

java.lang.Object

org.apache.hudi.io.HoodieIOHandle<T,I,K,O>

org.apache.hudi.io.HoodieWriteHandle<T,I,K,O>

org.apache.hudi.io.HoodieMergeHandle<T,I,K,O>

org.apache.hudi.io.HoodieSortedMergeHandle<T,I,K,O>

public class HoodieSortedMergeHandle<T extends HoodieRecordPayload,I,K,O>
extends HoodieMergeHandle<T,I,K,O>

Hoodie merge handle which writes records (new inserts or updates) sorted by their key. The implementation performs a merge-sort by comparing the key of the record being written to the list of keys in newRecordKeys (sorted in-memory).

Field Summary

Fields inherited from class org.apache.hudi.io.HoodieMergeHandle

fileWriter, insertRecordsWritten, keyGeneratorOpt, keyToNewRecords, newFilePath, oldFilePath, recordsDeleted, recordsWritten, updatedRecordsWritten, useWriterSchemaForCompaction, writtenRecordKeys

Fields inherited from class org.apache.hudi.io.HoodieWriteHandle

fileId, IGNORE_RECORD, partitionPath, schemaOnReadEnabled, tableSchema, tableSchemaWithMetaFields, taskContextSupplier, timer, writeSchema, writeSchemaWithMetaFields, writeStatus, writeToken

Fields inherited from class org.apache.hudi.io.HoodieIOHandle

config, fs, hoodieTable, instantTime

Constructor Summary

Constructors

Constructor and Description
HoodieSortedMergeHandle(HoodieWriteConfig config, String instantTime, HoodieTable<T,I,K,O> hoodieTable, Iterator<HoodieRecord<T>> recordItr, String partitionPath, String fileId, TaskContextSupplier taskContextSupplier, Option<BaseKeyGenerator> keyGeneratorOpt)
HoodieSortedMergeHandle(HoodieWriteConfig config, String instantTime, HoodieTable<T,I,K,O> hoodieTable, Map<String,HoodieRecord<T>> keyToNewRecordsOrig, String partitionPath, String fileId, HoodieBaseFile dataFileToBeMerged, TaskContextSupplier taskContextSupplier, Option<BaseKeyGenerator> keyGeneratorOpt) Called by compactor code path.

Method Summary

Modifier and Type	Method and Description
List<WriteStatus>	close()
void	write(org.apache.avro.generic.GenericRecord oldRecord) Go through an old record.

Methods inherited from class org.apache.hudi.io.HoodieMergeHandle

baseFileForMerge, getIOType, getOldFilePath, getWriterSchema, getWriterSchemaWithMetaFields, init, initializeIncomingRecordsMap, makeOldAndNewFilePaths, performMergeDataValidationCheck, setWriteStatusPath, writeIncomingRecords, writeInsertRecord, writeRecord, writeRecord, writeToFile

Methods inherited from class org.apache.hudi.io.HoodieWriteHandle

canWrite, createMarkerFile, createNewFileWriter, getAttemptId, getFileSystem, getPartitionId, getPartitionPath, getStageId, makeNewFilePath, makeNewPath, rewriteRecord, rewriteRecordWithMetadata, write, write, writeStatuses

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HoodieSortedMergeHandle

public HoodieSortedMergeHandle(HoodieWriteConfig config,
                               String instantTime,
                               HoodieTable<T,I,K,O> hoodieTable,
                               Iterator<HoodieRecord<T>> recordItr,
                               String partitionPath,
                               String fileId,
                               TaskContextSupplier taskContextSupplier,
                               Option<BaseKeyGenerator> keyGeneratorOpt)

HoodieSortedMergeHandle

public HoodieSortedMergeHandle(HoodieWriteConfig config,
                               String instantTime,
                               HoodieTable<T,I,K,O> hoodieTable,
                               Map<String,HoodieRecord<T>> keyToNewRecordsOrig,
                               String partitionPath,
                               String fileId,
                               HoodieBaseFile dataFileToBeMerged,
                               TaskContextSupplier taskContextSupplier,
                               Option<BaseKeyGenerator> keyGeneratorOpt)

Called by compactor code path.

Method Detail

write

public void write(org.apache.avro.generic.GenericRecord oldRecord)

Go through an old record. Here if we detect a newer version shows up, we write the new one to the file.

Overrides:

write in class HoodieMergeHandle<T extends HoodieRecordPayload,I,K,O>

close

public List<WriteStatus> close()

Overrides:

close in class HoodieMergeHandle<T extends HoodieRecordPayload,I,K,O>