org.apache.hudi.table.repair

Class RepairUtils

java.lang.Object

org.apache.hudi.table.repair.RepairUtils

public final class RepairUtils
extends Object

Utils for table repair tool.

Constructor Summary

Constructors

Constructor and Description
RepairUtils()

Method Summary

Modifier and Type	Method and Description
static List<String>	findInstantFilesToRemove(String instantToRepair, List<String> baseAndLogFilesFromFs, HoodieActiveTimeline activeTimeline, HoodieArchivedTimeline archivedTimeline) Finds the dangling files to remove for a given instant to repair.
static Option<Set<String>>	getBaseAndLogFilePathsFromTimeline(HoodieTimeline timeline, HoodieInstant instant) Gets the base and log file paths written for a given instant from the timeline.
static Map<String,List<String>>	tagInstantsOfBaseAndLogFiles(String basePath, List<StoragePath> allPaths) Tags the instant time of each base or log file from the input file paths.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

RepairUtils

public RepairUtils()

Method Detail

tagInstantsOfBaseAndLogFiles

public static Map<String,List<String>> tagInstantsOfBaseAndLogFiles(String basePath,
                                                                    List<StoragePath> allPaths)

Tags the instant time of each base or log file from the input file paths.

Parameters:

basePath - Base path of the table.

allPaths - A List of file paths to tag.

Returns:

A Map of instant time in String to a List of relative file paths.

getBaseAndLogFilePathsFromTimeline

public static Option<Set<String>> getBaseAndLogFilePathsFromTimeline(HoodieTimeline timeline,
                                                                     HoodieInstant instant)
                                                              throws IOException

Gets the base and log file paths written for a given instant from the timeline. This reads the details of the instant metadata.

Parameters:

timeline - HoodieTimeline instance, can be active or archived timeline.

instant - Instant for lookup.

Returns:

A Option of Set of relative file paths to base path if the instant action is supported; empty Option otherwise.

Throws:

IOException - if reading instant details fail.

findInstantFilesToRemove

public static List<String> findInstantFilesToRemove(String instantToRepair,
                                                    List<String> baseAndLogFilesFromFs,
                                                    HoodieActiveTimeline activeTimeline,
                                                    HoodieArchivedTimeline archivedTimeline)

Finds the dangling files to remove for a given instant to repair.

Parameters:

instantToRepair - Instant timestamp to repair.

baseAndLogFilesFromFs - A List of base and log files based on the file system.

activeTimeline - HoodieActiveTimeline instance.

archivedTimeline - HoodieArchivedTimeline instance.

Returns:

A List of relative file paths to base path for removing.