org.apache.hudi.sink.partitioner.profile

Class WriteProfiles

java.lang.Object

org.apache.hudi.sink.partitioner.profile.WriteProfiles

public class WriteProfiles
extends Object

Factory for WriteProfile.

Method Summary

Modifier and Type	Method and Description
static void	clean(String path)
static HoodieCommitMetadata	getCommitMetadata(String tableName, org.apache.flink.core.fs.Path basePath, HoodieInstant instant, HoodieTimeline timeline) Returns the commit metadata of the given instant.
static Option<HoodieCommitMetadata>	getCommitMetadataSafely(String tableName, org.apache.flink.core.fs.Path basePath, HoodieInstant instant, HoodieTimeline timeline) Returns the commit metadata of the given instant safely.
static org.apache.hadoop.fs.FileStatus[]	getRawWritePathsOfInstants(org.apache.flink.core.fs.Path basePath, org.apache.hadoop.conf.Configuration hadoopConf, List<HoodieCommitMetadata> metadataList, HoodieTableType tableType) Returns all the incremental write file statuses with the given commits metadata.
static org.apache.hadoop.fs.FileStatus[]	getWritePathsOfInstants(org.apache.flink.core.fs.Path basePath, org.apache.hadoop.conf.Configuration hadoopConf, List<HoodieCommitMetadata> metadataList, HoodieTableType tableType) Returns all the incremental write file statuses with the given commits metadata.
static WriteProfile	singleton(boolean ignoreSmallFiles, boolean delta, HoodieWriteConfig config, HoodieFlinkEngineContext context)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

singleton

public static WriteProfile singleton(boolean ignoreSmallFiles,
                                     boolean delta,
                                     HoodieWriteConfig config,
                                     HoodieFlinkEngineContext context)

clean

public static void clean(String path)

getRawWritePathsOfInstants

public static org.apache.hadoop.fs.FileStatus[] getRawWritePathsOfInstants(org.apache.flink.core.fs.Path basePath,
                                                                           org.apache.hadoop.conf.Configuration hadoopConf,
                                                                           List<HoodieCommitMetadata> metadataList,
                                                                           HoodieTableType tableType)

Returns all the incremental write file statuses with the given commits metadata.

Different with getWritePathsOfInstants(org.apache.flink.core.fs.Path, org.apache.hadoop.conf.Configuration, java.util.List<org.apache.hudi.common.model.HoodieCommitMetadata>, org.apache.hudi.common.model.HoodieTableType), the files are not filtered by existence.

Parameters:

basePath - Table base path

hadoopConf - The hadoop conf

metadataList - The commits metadata

tableType - The table type

Returns:

the file status array

getWritePathsOfInstants

public static org.apache.hadoop.fs.FileStatus[] getWritePathsOfInstants(org.apache.flink.core.fs.Path basePath,
                                                                        org.apache.hadoop.conf.Configuration hadoopConf,
                                                                        List<HoodieCommitMetadata> metadataList,
                                                                        HoodieTableType tableType)

Returns all the incremental write file statuses with the given commits metadata.

Parameters:

basePath - Table base path

hadoopConf - The hadoop conf

metadataList - The commits metadata

tableType - The table type

Returns:

the file status array

getCommitMetadataSafely

public static Option<HoodieCommitMetadata> getCommitMetadataSafely(String tableName,
                                                                   org.apache.flink.core.fs.Path basePath,
                                                                   HoodieInstant instant,
                                                                   HoodieTimeline timeline)

Returns the commit metadata of the given instant safely.

Parameters:

tableName - The table name

basePath - The table base path

instant - The hoodie instant

timeline - The timeline

Returns:

the commit metadata or empty if any error occurs

getCommitMetadata

public static HoodieCommitMetadata getCommitMetadata(String tableName,
                                                     org.apache.flink.core.fs.Path basePath,
                                                     HoodieInstant instant,
                                                     HoodieTimeline timeline)

Returns the commit metadata of the given instant.

Parameters:

tableName - The table name

basePath - The table base path

instant - The hoodie instant

timeline - The timeline

Returns:

the commit metadata