| Constructor and Description |
|---|
BaseHoodieTableFileIndex(HoodieEngineContext engineContext,
HoodieTableMetaClient metaClient,
TypedProperties configProperties,
HoodieTableQueryType queryType,
List<StoragePath> queryPaths,
Option<String> specifiedQueryInstant,
boolean shouldIncludePendingCommits,
boolean shouldValidateInstant,
BaseHoodieTableFileIndex.FileStatusCache fileStatusCache,
boolean shouldListLazily,
Option<String> beginInstantTime,
Option<String> endInstantTime) |
| Modifier and Type | Field and Description |
|---|---|
protected TypedProperties |
HoodieConfig.props |
| Modifier and Type | Method and Description |
|---|---|
static TypedProperties |
TypedProperties.fromMap(Map<?,?> items)
This method is introduced to get rid of the scala compile error:
|
TypedProperties |
LockConfiguration.getConfig() |
TypedProperties |
PropertiesConfig.getGlobalProperties()
return any global properties for the filesystem
|
TypedProperties |
HoodieConfig.getProps() |
TypedProperties |
HoodieConfig.getProps(boolean includeGlobalProps) |
| Modifier and Type | Method and Description |
|---|---|
static void |
TypedProperties.putAll(TypedProperties props,
Map<?,?> items)
This method is introduced to get rid of the scala compile error:
|
| Constructor and Description |
|---|
HoodieConfig(TypedProperties props) |
| Modifier and Type | Method and Description |
|---|---|
Comparable |
HoodieReaderContext.getOrderingValue(Option<T> recordOption,
Map<String,Object> metadataMap,
org.apache.avro.Schema schema,
TypedProperties props)
Gets the ordering value in particular type.
|
| Modifier and Type | Method and Description |
|---|---|
default List<Pair<HoodieRecord,org.apache.avro.Schema>> |
HoodieRecordMerger.fullOuterMerge(HoodieRecord older,
org.apache.avro.Schema oldSchema,
HoodieRecord newer,
org.apache.avro.Schema newSchema,
TypedProperties props)
Merges two records with the same key in full outer merge fashion i.e.
|
List<Pair<HoodieRecord,org.apache.avro.Schema>> |
HoodieMetadataRecordMerger.fullOuterMerge(HoodieRecord older,
org.apache.avro.Schema oldSchema,
HoodieRecord newer,
org.apache.avro.Schema newSchema,
TypedProperties props) |
Option<Pair<HoodieRecord,org.apache.avro.Schema>> |
HoodiePreCombineAvroRecordMerger.merge(HoodieRecord older,
org.apache.avro.Schema oldSchema,
HoodieRecord newer,
org.apache.avro.Schema newSchema,
TypedProperties props) |
Option<Pair<HoodieRecord,org.apache.avro.Schema>> |
HoodieAvroRecordMerger.merge(HoodieRecord older,
org.apache.avro.Schema oldSchema,
HoodieRecord newer,
org.apache.avro.Schema newSchema,
TypedProperties props) |
Option<Pair<HoodieRecord,org.apache.avro.Schema>> |
HoodieRecordMerger.merge(HoodieRecord older,
org.apache.avro.Schema oldSchema,
HoodieRecord newer,
org.apache.avro.Schema newSchema,
TypedProperties props)
This method converges combineAndGetUpdateValue and precombine from HoodiePayload.
|
Option<Pair<HoodieRecord,org.apache.avro.Schema>> |
OverwriteWithLatestMerger.merge(HoodieRecord older,
org.apache.avro.Schema oldSchema,
HoodieRecord newer,
org.apache.avro.Schema newSchema,
TypedProperties props) |
default Option<Pair<HoodieRecord,org.apache.avro.Schema>> |
HoodieRecordMerger.partialMerge(HoodieRecord older,
org.apache.avro.Schema oldSchema,
HoodieRecord newer,
org.apache.avro.Schema newSchema,
org.apache.avro.Schema readerSchema,
TypedProperties props)
Merges records which can contain partial updates, i.e., only subset of fields and values are
present in the record representing the updates, and absent fields are not updated.
|
default boolean |
HoodieRecordMerger.shouldFlush(HoodieRecord record,
org.apache.avro.Schema schema,
TypedProperties props)
In some cases a business logic does some checks before flushing a merged record to the disk.
|
| Modifier and Type | Method and Description |
|---|---|
protected TypedProperties |
AbstractHoodieLogRecordReader.getPayloadProps() |
protected TypedProperties |
BaseHoodieLogRecordReader.getPayloadProps() |
| Modifier and Type | Field and Description |
|---|---|
protected TypedProperties |
HoodieBaseFileGroupRecordBuffer.props |
| Constructor and Description |
|---|
HoodieBaseFileGroupRecordBuffer(HoodieReaderContext<T> readerContext,
HoodieTableMetaClient hoodieTableMetaClient,
Option<String> partitionNameOverrideOpt,
Option<String[]> partitionPathFieldOpt,
HoodieRecordMerger recordMerger,
TypedProperties props) |
HoodieFileGroupReader(HoodieReaderContext<T> readerContext,
HoodieStorage storage,
String tablePath,
String latestCommitTime,
FileSlice fileSlice,
org.apache.avro.Schema dataSchema,
org.apache.avro.Schema requestedSchema,
Option<InternalSchema> internalSchemaOpt,
HoodieTableMetaClient hoodieTableMetaClient,
TypedProperties props,
long start,
long length,
boolean shouldUseRecordPosition) |
HoodieKeyBasedFileGroupRecordBuffer(HoodieReaderContext<T> readerContext,
HoodieTableMetaClient hoodieTableMetaClient,
Option<String> partitionNameOverrideOpt,
Option<String[]> partitionPathFieldOpt,
HoodieRecordMerger recordMerger,
TypedProperties props) |
HoodiePositionBasedFileGroupRecordBuffer(HoodieReaderContext<T> readerContext,
HoodieTableMetaClient hoodieTableMetaClient,
Option<String> partitionNameOverrideOpt,
Option<String[]> partitionPathFieldOpt,
HoodieRecordMerger recordMerger,
TypedProperties props) |
HoodieUnmergedFileGroupRecordBuffer(HoodieReaderContext<T> readerContext,
HoodieTableMetaClient hoodieTableMetaClient,
Option<String> partitionNameOverrideOpt,
Option<String[]> partitionPathFieldOpt,
HoodieRecordMerger recordMerger,
TypedProperties props) |
| Modifier and Type | Method and Description |
|---|---|
static TypedProperties |
ConfigUtils.fetchConfigs(HoodieStorage storage,
StoragePath metaPath,
String propertiesFile,
String propertiesBackupFile,
int maxReadRetries,
int maxReadRetryDelayInMs) |
static TypedProperties |
ConfigUtils.loadGlobalProperties() |
| Modifier and Type | Method and Description |
|---|---|
static void |
ConfigUtils.checkRequiredConfigProperties(TypedProperties props,
List<ConfigProperty<?>> configPropertyList)
Validates that all
ConfigProperty configs exist in the properties. |
static void |
ConfigUtils.checkRequiredProperties(TypedProperties props,
List<String> checkPropNames)
Validates that config String keys exist in the properties.
|
static boolean |
ConfigUtils.containsConfigProperty(TypedProperties props,
ConfigProperty<?> configProperty)
Whether the properties contain a config.
|
static Map<String,Object> |
ConfigUtils.filterProperties(TypedProperties props,
List<ConfigProperty<String>> configPropertyList)
Returns filtered properties based on the given
ConfigProperty config list to keep. |
static int |
ConfigUtils.getIntWithAltKeys(TypedProperties props,
ConfigProperty<?> configProperty)
Gets the integer value for a
ConfigProperty config from properties. |
static long |
ConfigUtils.getLongWithAltKeys(TypedProperties props,
ConfigProperty<Long> configProperty)
Gets the long value for a
ConfigProperty config from properties. |
static String |
ConfigUtils.getStringWithAltKeys(TypedProperties props,
String key,
String altKey,
String defaultValue)
Gets String value from properties with alternative keys.
|
static void |
ConfigUtils.removeConfigFromProps(TypedProperties props,
ConfigProperty<?> configProperty)
Removes a
ConfigProperty config from properties. |
| Modifier and Type | Field and Description |
|---|---|
protected TypedProperties |
KeyGenerator.config |
| Constructor and Description |
|---|
BaseKeyGenerator(TypedProperties config) |
KeyGenerator(TypedProperties config) |
| Modifier and Type | Method and Description |
|---|---|
static List<MetadataPartitionType> |
MetadataPartitionType.getEnabledPartitions(TypedProperties writeConfig,
HoodieTableMetaClient metaClient)
Returns the list of metadata partition types enabled based on the metadata config and table config.
|
abstract boolean |
MetadataPartitionType.isMetadataPartitionEnabled(TypedProperties writeConfig)
Check if the metadata partition is enabled based on the metadata config.
|
Copyright © 2024 The Apache Software Foundation. All rights reserved.