Uses of Class
org.apache.hudi.common.engine.HoodieEngineContext

Packages that use HoodieEngineContext

Package	Description
org.apache.hudi
org.apache.hudi.common.data
org.apache.hudi.common.engine
org.apache.hudi.common.fs
org.apache.hudi.common.table.view
org.apache.hudi.common.util
org.apache.hudi.metadata

Uses of HoodieEngineContext in org.apache.hudi

Constructors in org.apache.hudi with parameters of type HoodieEngineContext

Constructor and Description

BaseHoodieTableFileIndex(HoodieEngineContext engineContext, HoodieTableMetaClient metaClient, TypedProperties configProperties, HoodieTableQueryType queryType, List<StoragePath> queryPaths, Option<String> specifiedQueryInstant, boolean shouldIncludePendingCommits, boolean shouldValidateInstant, BaseHoodieTableFileIndex.FileStatusCache fileStatusCache, boolean shouldListLazily, Option<String> beginInstantTime, Option<String> endInstantTime)

Uses of HoodieEngineContext in org.apache.hudi.common.data

Methods in org.apache.hudi.common.data with parameters of type HoodieEngineContext

Modifier and Type	Method and Description
void	HoodieData.persist(String level, HoodieEngineContext engineContext, HoodieData.HoodieDataCacheKey cacheKey) Persists the data w/ provided level (if applicable), and cache the data's ids within the engineContext.
void	HoodieListData.persist(String level, HoodieEngineContext engineContext, HoodieData.HoodieDataCacheKey cacheKey)

Uses of HoodieEngineContext in org.apache.hudi.common.engine

Subclasses of HoodieEngineContext in org.apache.hudi.common.engine

Modifier and Type	Class and Description
class	HoodieLocalEngineContext A java based engine context, use this implementation on the query engine integrations if needed.

Uses of HoodieEngineContext in org.apache.hudi.common.fs

Methods in org.apache.hudi.common.fs with parameters of type HoodieEngineContext

Modifier and Type	Method and Description
static boolean	FSUtils.deleteDir(HoodieEngineContext hoodieEngineContext, HoodieStorage storage, StoragePath dirPath, int parallelism) Deletes a directory by deleting sub-paths in parallel on the file system.
static List<String>	FSUtils.getAllPartitionPaths(HoodieEngineContext engineContext, HoodieStorage storage, HoodieMetadataConfig metadataConfig, StoragePath basePath)
static List<String>	FSUtils.getAllPartitionPaths(HoodieEngineContext engineContext, HoodieStorage storage, HoodieMetadataConfig metadataConfig, String basePathStr)
static List<String>	FSUtils.getAllPartitionPaths(HoodieEngineContext engineContext, HoodieStorage storage, StoragePath basePath, boolean useFileListingFromMetadata)
static List<String>	FSUtils.getAllPartitionPaths(HoodieEngineContext engineContext, HoodieStorage storage, String basePathStr, boolean useFileListingFromMetadata)
static Map<String,List<StoragePathInfo>>	FSUtils.getFilesInPartitions(HoodieEngineContext engineContext, HoodieStorage storage, HoodieMetadataConfig metadataConfig, String basePathStr, String[] partitionPaths)
static <T> Map<String,T>	FSUtils.parallelizeFilesProcess(HoodieEngineContext hoodieEngineContext, HoodieStorage storage, int parallelism, FSUtils.SerializableFunction<Pair<String,StorageConfiguration<?>>,T> pairFunction, List<String> subPaths)
static <T> Map<String,T>	FSUtils.parallelizeSubPathProcess(HoodieEngineContext hoodieEngineContext, HoodieStorage storage, StoragePath dirPath, int parallelism, Predicate<StoragePathInfo> subPathPredicate, FSUtils.SerializableFunction<Pair<String,StorageConfiguration<?>>,T> pairFunction) Processes sub-path in parallel.

Uses of HoodieEngineContext in org.apache.hudi.common.table.view

Methods in org.apache.hudi.common.table.view with parameters of type HoodieEngineContext

Modifier and Type	Method and Description
static HoodieTableFileSystemView	FileSystemViewManager.createInMemoryFileSystemView(HoodieEngineContext engineContext, HoodieTableMetaClient metaClient, HoodieMetadataConfig metadataConfig)
static HoodieTableFileSystemView	FileSystemViewManager.createInMemoryFileSystemViewWithTimeline(HoodieEngineContext engineContext, HoodieTableMetaClient metaClient, HoodieMetadataConfig metadataConfig, HoodieTimeline timeline)
static FileSystemViewManager	FileSystemViewManager.createViewManager(HoodieEngineContext context, FileSystemViewStorageConfig config, HoodieCommonConfig commonConfig)
static FileSystemViewManager	FileSystemViewManager.createViewManager(HoodieEngineContext context, FileSystemViewStorageConfig config, HoodieCommonConfig commonConfig, SerializableFunctionUnchecked<HoodieTableMetaClient,HoodieTableMetadata> metadataCreator) Main Factory method for building file-system views.
static FileSystemViewManager	FileSystemViewManager.createViewManagerWithTableMetadata(HoodieEngineContext context, HoodieMetadataConfig metadataConfig, FileSystemViewStorageConfig config, HoodieCommonConfig commonConfig)

Uses of HoodieEngineContext in org.apache.hudi.common.util

Methods in org.apache.hudi.common.util with parameters of type HoodieEngineContext

Modifier and Type	Method and Description
static Map<String,Set<String>>	MarkerUtils.readTimelineServerBasedMarkersFromFileSystem(String markerDir, HoodieStorage storage, HoodieEngineContext context, int parallelism) Reads files containing the markers written by timeline-server-based marker mechanism.

Uses of HoodieEngineContext in org.apache.hudi.metadata

Fields in org.apache.hudi.metadata declared as HoodieEngineContext

Modifier and Type	Field and Description
protected HoodieEngineContext	AbstractHoodieTableMetadata.engineContext

Methods in org.apache.hudi.metadata that return HoodieEngineContext

Modifier and Type	Method and Description
protected HoodieEngineContext	BaseTableMetadata.getEngineContext()

Methods in org.apache.hudi.metadata with parameters of type HoodieEngineContext

Modifier and Type	Method and Description
static HoodieData<HoodieRecord>	HoodieTableMetadataUtil.convertFilesToBloomFilterRecords(HoodieEngineContext engineContext, Map<String,List<String>> partitionToDeletedFiles, Map<String,Map<String,Long>> partitionToAppendedFiles, String instantTime, HoodieTableMetaClient dataMetaClient, int bloomIndexParallelism, String bloomFilterType) Convert added and deleted files metadata to bloom filter index records.
static HoodieData<HoodieRecord>	HoodieTableMetadataUtil.convertFilesToColumnStatsRecords(HoodieEngineContext engineContext, Map<String,List<String>> partitionToDeletedFiles, Map<String,Map<String,Long>> partitionToAppendedFiles, HoodieTableMetaClient dataMetaClient, boolean isColumnStatsIndexEnabled, int columnStatsIndexParallelism, List<String> targetColumnsForColumnStatsIndex) Convert added and deleted action metadata to column stats index records.
static HoodieData<HoodieRecord>	HoodieTableMetadataUtil.convertFilesToPartitionStatsRecords(HoodieEngineContext engineContext, List<HoodieTableMetadataUtil.DirectoryInfo> partitionInfoList, HoodieMetadataConfig metadataConfig, HoodieTableMetaClient dataTableMetaClient)
static HoodieData<HoodieRecord>	HoodieTableMetadataUtil.convertMetadataToBloomFilterRecords(HoodieCleanMetadata cleanMetadata, HoodieEngineContext engineContext, String instantTime, int bloomIndexParallelism) Convert clean metadata to bloom filter index records.
static HoodieData<HoodieRecord>	HoodieTableMetadataUtil.convertMetadataToBloomFilterRecords(HoodieEngineContext context, HoodieConfig hoodieConfig, HoodieCommitMetadata commitMetadata, String instantTime, HoodieTableMetaClient dataMetaClient, String bloomFilterType, int bloomIndexParallelism) Convert commit action metadata to bloom filter records.
static HoodieData<HoodieRecord>	HoodieTableMetadataUtil.convertMetadataToColumnStatsRecords(HoodieCleanMetadata cleanMetadata, HoodieEngineContext engineContext, HoodieTableMetaClient dataMetaClient, boolean isColumnStatsIndexEnabled, int columnStatsIndexParallelism, List<String> targetColumnsForColumnStatsIndex) Convert clean metadata to column stats index records.
static HoodieData<HoodieRecord>	HoodieTableMetadataUtil.convertMetadataToColumnStatsRecords(HoodieCommitMetadata commitMetadata, HoodieEngineContext engineContext, HoodieTableMetaClient dataMetaClient, boolean isColumnStatsIndexEnabled, int columnStatsIndexParallelism, List<String> targetColumnsForColumnStatsIndex)
static HoodieData<HoodieRecord>	HoodieTableMetadataUtil.convertMetadataToPartitionStatsRecords(HoodieCommitMetadata commitMetadata, HoodieEngineContext engineContext, HoodieTableMetaClient dataMetaClient, HoodieMetadataConfig metadataConfig)
static Map<MetadataPartitionType,HoodieData<HoodieRecord>>	HoodieTableMetadataUtil.convertMetadataToRecords(HoodieEngineContext engineContext, HoodieCleanMetadata cleanMetadata, String instantTime, HoodieTableMetaClient dataMetaClient, List<MetadataPartitionType> enabledPartitionTypes, int bloomIndexParallelism, boolean isColumnStatsIndexEnabled, int columnStatsIndexParallelism, List<String> targetColumnsForColumnStatsIndex) Convert the clean action to metadata records.
static Map<MetadataPartitionType,HoodieData<HoodieRecord>>	HoodieTableMetadataUtil.convertMetadataToRecords(HoodieEngineContext context, HoodieConfig hoodieConfig, HoodieCommitMetadata commitMetadata, String instantTime, HoodieTableMetaClient dataMetaClient, List<MetadataPartitionType> enabledPartitionTypes, String bloomFilterType, int bloomIndexParallelism, boolean isColumnStatsIndexEnabled, int columnStatsIndexParallelism, List<String> targetColumnsForColumnStatsIndex, HoodieMetadataConfig metadataConfig) Convert commit action to metadata records for the enabled partition types.
static Map<MetadataPartitionType,HoodieData<HoodieRecord>>	HoodieTableMetadataUtil.convertMetadataToRecords(HoodieEngineContext engineContext, HoodieTableMetaClient dataTableMetaClient, HoodieRollbackMetadata rollbackMetadata, String instantTime) Convert rollback action metadata to metadata table records.
static Map<MetadataPartitionType,HoodieData<HoodieRecord>>	HoodieTableMetadataUtil.convertMissingPartitionRecords(HoodieEngineContext engineContext, List<String> deletedPartitions, Map<String,Map<String,Long>> filesAdded, Map<String,List<String>> filesDeleted, String instantTime)
static HoodieTableMetadata	HoodieTableMetadata.create(HoodieEngineContext engineContext, HoodieStorage storage, HoodieMetadataConfig metadataConfig, String datasetBasePath)
static HoodieTableMetadata	HoodieTableMetadata.create(HoodieEngineContext engineContext, HoodieStorage storage, HoodieMetadataConfig metadataConfig, String datasetBasePath, boolean reuse)
static FileSystemBackedTableMetadata	HoodieTableMetadata.createFSBackedTableMetadata(HoodieEngineContext engineContext, HoodieStorage storage, String datasetBasePath)
static HoodieBackedTableMetadata	HoodieTableMetadata.createHoodieBackedTableMetadata(HoodieEngineContext engineContext, HoodieStorage storage, HoodieMetadataConfig metadataConfig, String datasetBasePath, boolean reuse)
static void	HoodieTableMetadataUtil.deleteMetadataPartition(StoragePath basePath, HoodieEngineContext context, String partitionPath) Deletes the metadata partition from the file system.
static String	HoodieTableMetadataUtil.deleteMetadataTable(HoodieTableMetaClient dataMetaClient, HoodieEngineContext context, boolean backup) Delete the metadata table for the dataset and backup if required.
static void	HoodieTableMetadataUtil.deleteMetadataTable(String basePath, HoodieEngineContext context) Delete the metadata table for the dataset.
static String	HoodieTableMetadataUtil.deleteMetadataTablePartition(HoodieTableMetaClient dataMetaClient, HoodieEngineContext context, String partitionPath, boolean backup) Delete a partition within the metadata table.
static boolean	HoodieTableMetadataUtil.metadataPartitionExists(StoragePath basePath, HoodieEngineContext context, String partitionPath)
static boolean	HoodieTableMetadataUtil.metadataPartitionExists(String basePath, HoodieEngineContext context, String partitionPath) Check if the given metadata partition exists.
static HoodieData<HoodieRecord>	HoodieTableMetadataUtil.readRecordKeysFromBaseFiles(HoodieEngineContext engineContext, HoodieConfig config, List<Pair<String,HoodieBaseFile>> partitionBaseFilePairs, boolean forDelete, int recordIndexMaxParallelism, StoragePath basePath, StorageConfiguration<?> configuration, String activeModule) Deprecated.
static HoodieData<HoodieRecord>	HoodieTableMetadataUtil.readRecordKeysFromFileSlices(HoodieEngineContext engineContext, List<Pair<String,FileSlice>> partitionFileSlicePairs, boolean forDelete, int recordIndexMaxParallelism, String activeModule, HoodieTableMetaClient metaClient, EngineType engineType) Reads the record keys from the given file slices and returns a HoodieData of HoodieRecord to be updated in the metadata table.
static HoodieData<HoodieRecord>	HoodieTableMetadataUtil.readSecondaryKeysFromBaseFiles(HoodieEngineContext engineContext, List<Pair<String,Pair<String,List<String>>>> partitionFiles, int secondaryIndexMaxParallelism, String activeModule, HoodieTableMetaClient metaClient, EngineType engineType, HoodieIndexDefinition indexDefinition)
static HoodieData<HoodieRecord>	HoodieTableMetadataUtil.readSecondaryKeysFromFileSlices(HoodieEngineContext engineContext, List<Pair<String,FileSlice>> partitionFileSlicePairs, int secondaryIndexMaxParallelism, String activeModule, HoodieTableMetaClient metaClient, EngineType engineType, HoodieIndexDefinition indexDefinition)

Constructors in org.apache.hudi.metadata with parameters of type HoodieEngineContext

Constructor and Description
AbstractHoodieTableMetadata(HoodieEngineContext engineContext, HoodieStorage storage, String dataBasePath)
BaseTableMetadata(HoodieEngineContext engineContext, HoodieStorage storage, HoodieMetadataConfig metadataConfig, String dataBasePath)
FileSystemBackedTableMetadata(HoodieEngineContext engineContext, HoodieStorage storage, String datasetBasePath)
FileSystemBackedTableMetadata(HoodieEngineContext engineContext, HoodieTableConfig tableConfig, HoodieStorage storage, String datasetBasePath)
HoodieBackedTableMetadata(HoodieEngineContext engineContext, HoodieStorage storage, HoodieMetadataConfig metadataConfig, String datasetBasePath)
HoodieBackedTableMetadata(HoodieEngineContext engineContext, HoodieStorage storage, HoodieMetadataConfig metadataConfig, String datasetBasePath, boolean reuse)
HoodieMetadataFileSystemView(HoodieEngineContext engineContext, HoodieTableMetaClient metaClient, HoodieTimeline visibleActiveTimeline, HoodieMetadataConfig metadataConfig)