org.apache.hudi.common.model

Class HoodieFileGroup

java.lang.Object

org.apache.hudi.common.model.HoodieFileGroup

All Implemented Interfaces:

Serializable

public class HoodieFileGroup
extends Object
implements Serializable

A set of data/base files + set of log files, that make up a unit for all operations.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
HoodieFileGroup(HoodieFileGroup fileGroup)
HoodieFileGroup(HoodieFileGroupId fileGroupId, HoodieTimeline timeline)
HoodieFileGroup(String partitionPath, String id, HoodieTimeline timeline)

Method Summary

Modifier and Type	Method and Description
void	addBaseFile(HoodieBaseFile dataFile) Add a new datafile into the file group.
void	addFileSlice(FileSlice slice)
void	addLogFile(CompletionTimeQueryView completionTimeQueryView, HoodieLogFile logFile) Add a new log file into the group.
void	addNewFileSliceAtInstant(String baseInstantTime) Potentially add a new file-slice by adding base-instant time A file-slice without any data-file and log-files can exist (if a compaction just got requested).
Stream<HoodieBaseFile>	getAllBaseFiles() Stream of committed data files, sorted reverse commit time.
Stream<FileSlice>	getAllFileSlices() Provides a stream of committed file slices, sorted reverse base commit time.
Stream<FileSlice>	getAllFileSlicesBeforeOn(String maxInstantTime)
Stream<FileSlice>	getAllFileSlicesIncludingInflight() Get all the file slices including in-flight ones as seen in underlying file system.
Stream<FileSlice>	getAllRawFileSlices()
String	getBaseInstantTime(CompletionTimeQueryView completionTimeQueryView, HoodieLogFile logFile)
HoodieFileGroupId	getFileGroupId()
Option<HoodieBaseFile>	getLatestDataFile() Gets the latest data file.
Option<FileSlice>	getLatestFileSlice() Gets the latest slice - this can contain either.
Option<FileSlice>	getLatestFileSliceBefore(String maxInstantTime) Obtain the latest file slice, upto an instantTime i.e < maxInstantTime.
Option<FileSlice>	getLatestFileSliceBeforeOrOn(String maxInstantTime) Obtain the latest file slice, upto a instantTime i.e <= maxInstantTime.
Option<FileSlice>	getLatestFileSliceInRange(List<String> commitRange)
Option<FileSlice>	getLatestFileSlicesIncludingInflight() Get the latest file slices including inflight ones.
String	getPartitionPath()
static Comparator<String>	getReverseCommitTimeComparator()
HoodieTimeline	getTimeline()
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

HoodieFileGroup

public HoodieFileGroup(HoodieFileGroup fileGroup)

HoodieFileGroup

public HoodieFileGroup(String partitionPath,
                       String id,
                       HoodieTimeline timeline)

HoodieFileGroup

public HoodieFileGroup(HoodieFileGroupId fileGroupId,
                       HoodieTimeline timeline)

Method Detail

getReverseCommitTimeComparator

public static Comparator<String> getReverseCommitTimeComparator()

addNewFileSliceAtInstant

public void addNewFileSliceAtInstant(String baseInstantTime)

Potentially add a new file-slice by adding base-instant time A file-slice without any data-file and log-files can exist (if a compaction just got requested).

addBaseFile

public void addBaseFile(HoodieBaseFile dataFile)

Add a new datafile into the file group.

addLogFile

public void addLogFile(CompletionTimeQueryView completionTimeQueryView,
                       HoodieLogFile logFile)

Add a new log file into the group.

CAUTION: the log file must be added in sequence of the delta commit time.

getBaseInstantTime

public String getBaseInstantTime(CompletionTimeQueryView completionTimeQueryView,
                                 HoodieLogFile logFile)

getPartitionPath

public String getPartitionPath()

getFileGroupId

public HoodieFileGroupId getFileGroupId()

getAllFileSlicesIncludingInflight

public Stream<FileSlice> getAllFileSlicesIncludingInflight()

Get all the file slices including in-flight ones as seen in underlying file system.

getLatestFileSlicesIncludingInflight

public Option<FileSlice> getLatestFileSlicesIncludingInflight()

Get the latest file slices including inflight ones.

getAllFileSlices

public Stream<FileSlice> getAllFileSlices()

Provides a stream of committed file slices, sorted reverse base commit time.

getAllFileSlicesBeforeOn

public Stream<FileSlice> getAllFileSlicesBeforeOn(String maxInstantTime)

getLatestFileSlice

public Option<FileSlice> getLatestFileSlice()

Gets the latest slice - this can contain either.

- just the log files without data file - (or) data file with 0 or more log files

getLatestDataFile

public Option<HoodieBaseFile> getLatestDataFile()

Gets the latest data file.

getLatestFileSliceBeforeOrOn

public Option<FileSlice> getLatestFileSliceBeforeOrOn(String maxInstantTime)

Obtain the latest file slice, upto a instantTime i.e <= maxInstantTime.

getLatestFileSliceBefore

public Option<FileSlice> getLatestFileSliceBefore(String maxInstantTime)

Obtain the latest file slice, upto an instantTime i.e < maxInstantTime.

Parameters:

maxInstantTime - Max Instant Time

Returns:

the latest file slice

getLatestFileSliceInRange

public Option<FileSlice> getLatestFileSliceInRange(List<String> commitRange)

getAllBaseFiles

public Stream<HoodieBaseFile> getAllBaseFiles()

Stream of committed data files, sorted reverse commit time.

toString

public String toString()

Overrides:

toString in class Object

addFileSlice

public void addFileSlice(FileSlice slice)

getAllRawFileSlices

public Stream<FileSlice> getAllRawFileSlices()

getTimeline

public HoodieTimeline getTimeline()