Uses of Class
org.apache.hudi.internal.schema.InternalSchema

Packages that use InternalSchema

Package	Description
org.apache.hudi.common.table
org.apache.hudi.common.table.log
org.apache.hudi.common.table.log.block
org.apache.hudi.common.table.read
org.apache.hudi.common.util
org.apache.hudi.internal.schema
org.apache.hudi.internal.schema.action
org.apache.hudi.internal.schema.convert
org.apache.hudi.internal.schema.io
org.apache.hudi.internal.schema.utils
org.apache.hudi.internal.schema.visitor

Uses of InternalSchema in org.apache.hudi.common.table

Methods in org.apache.hudi.common.table that return types with arguments of type InternalSchema

Modifier and Type	Method and Description
Option<InternalSchema>	TableSchemaResolver.getTableInternalSchemaFromCommitMetadata() Gets the InternalSchema for a hoodie table from the HoodieCommitMetadata of the instant.
Option<InternalSchema>	TableSchemaResolver.getTableInternalSchemaFromCommitMetadata(String timestamp) Gets the InternalSchema for a hoodie table from the HoodieCommitMetadata of the instant.

Uses of InternalSchema in org.apache.hudi.common.table.log

Methods in org.apache.hudi.common.table.log with parameters of type InternalSchema

Modifier and Type	Method and Description
abstract AbstractHoodieLogRecordReader.Builder	AbstractHoodieLogRecordReader.Builder.withInternalSchema(InternalSchema internalSchema)
HoodieMetadataMergedLogRecordScanner.Builder	HoodieMetadataMergedLogRecordScanner.Builder.withInternalSchema(InternalSchema internalSchema)
HoodieMergedLogRecordScanner.Builder	HoodieMergedLogRecordScanner.Builder.withInternalSchema(InternalSchema internalSchema)
HoodieUnMergedLogRecordScanner.Builder	HoodieUnMergedLogRecordScanner.Builder.withInternalSchema(InternalSchema internalSchema)

Constructors in org.apache.hudi.common.table.log with parameters of type InternalSchema

Constructor and Description
AbstractHoodieLogRecordReader(HoodieStorage storage, String basePath, List<String> logFilePaths, org.apache.avro.Schema readerSchema, String latestInstantTime, boolean reverseReader, int bufferSize, Option<InstantRange> instantRange, boolean withOperationField, boolean forceFullScan, Option<String> partitionNameOverride, InternalSchema internalSchema, Option<String> keyFieldOverride, boolean enableOptimizedLogBlocksScan, HoodieRecordMerger recordMerger, Option<HoodieTableMetaClient> hoodieTableMetaClientOption)
BaseHoodieMergedLogRecordScanner(HoodieStorage storage, String basePath, List<String> logFilePaths, org.apache.avro.Schema readerSchema, String latestInstantTime, Long maxMemorySizeInBytes, boolean reverseReader, int bufferSize, String spillableMapBasePath, Option<InstantRange> instantRange, ExternalSpillableMap.DiskMapType diskMapType, boolean isBitCaskDiskMapCompressionEnabled, boolean withOperationField, boolean forceFullScan, Option<String> partitionName, InternalSchema internalSchema, Option<String> keyFieldOverride, boolean enableOptimizedLogBlocksScan, HoodieRecordMerger recordMerger, Option<HoodieTableMetaClient> hoodieTableMetaClientOption)
HoodieLogFileReader(HoodieStorage storage, HoodieLogFile logFile, org.apache.avro.Schema readerSchema, int bufferSize, boolean reverseReader, boolean enableRecordLookups, String keyField, InternalSchema internalSchema)
HoodieMergedLogRecordScanner(HoodieStorage storage, String basePath, List<String> logFilePaths, org.apache.avro.Schema readerSchema, String latestInstantTime, Long maxMemorySizeInBytes, boolean reverseReader, int bufferSize, String spillableMapBasePath, Option<InstantRange> instantRange, ExternalSpillableMap.DiskMapType diskMapType, boolean isBitCaskDiskMapCompressionEnabled, boolean withOperationField, boolean forceFullScan, Option<String> partitionName, InternalSchema internalSchema, Option<String> keyFieldOverride, boolean enableOptimizedLogBlocksScan, HoodieRecordMerger recordMerger, Option<HoodieTableMetaClient> hoodieTableMetaClientOption)

Uses of InternalSchema in org.apache.hudi.common.table.log.block

Methods in org.apache.hudi.common.table.log.block with parameters of type InternalSchema

Modifier and Type	Method and Description
static HoodieAvroDataBlock	HoodieAvroDataBlock.getBlock(byte[] content, org.apache.avro.Schema readerSchema, InternalSchema internalSchema) Deprecated.

Uses of InternalSchema in org.apache.hudi.common.table.read

Fields in org.apache.hudi.common.table.read declared as InternalSchema

Modifier and Type	Field and Description
protected InternalSchema	HoodieFileGroupReaderSchemaHandler.internalSchema
protected InternalSchema	HoodieBaseFileGroupRecordBuffer.internalSchema

Methods in org.apache.hudi.common.table.read that return InternalSchema

Modifier and Type	Method and Description
InternalSchema	HoodieFileGroupReaderSchemaHandler.getInternalSchema()

Constructor parameters in org.apache.hudi.common.table.read with type arguments of type InternalSchema

Constructor and Description
HoodieFileGroupReader(HoodieReaderContext<T> readerContext, HoodieStorage storage, String tablePath, String latestCommitTime, FileSlice fileSlice, org.apache.avro.Schema dataSchema, org.apache.avro.Schema requestedSchema, Option<InternalSchema> internalSchemaOpt, HoodieTableMetaClient hoodieTableMetaClient, TypedProperties props, long start, long length, boolean shouldUseRecordPosition)
HoodieFileGroupReaderSchemaHandler(HoodieReaderContext<T> readerContext, org.apache.avro.Schema dataSchema, org.apache.avro.Schema requestedSchema, Option<InternalSchema> internalSchemaOpt, HoodieTableConfig hoodieTableConfig)
HoodiePositionBasedSchemaHandler(HoodieReaderContext<T> readerContext, org.apache.avro.Schema dataSchema, org.apache.avro.Schema requestedSchema, Option<InternalSchema> internalSchemaOpt, HoodieTableConfig hoodieTableConfig)

Uses of InternalSchema in org.apache.hudi.common.util

Methods in org.apache.hudi.common.util that return InternalSchema

Modifier and Type	Method and Description
static InternalSchema	InternalSchemaCache.getInternalSchemaByVersionId(long versionId, HoodieTableMetaClient metaClient)
static InternalSchema	InternalSchemaCache.getInternalSchemaByVersionId(long versionId, String tablePath, HoodieStorage storage, String validCommits) Give a schema versionId return its internalSchema.
static InternalSchema	InternalSchemaCache.searchSchemaAndCache(long versionID, HoodieTableMetaClient metaClient) Search internalSchema based on versionID.

Uses of InternalSchema in org.apache.hudi.internal.schema

Methods in org.apache.hudi.internal.schema that return InternalSchema

Modifier and Type	Method and Description
static InternalSchema	InternalSchema.getEmptyInternalSchema()
InternalSchema	InternalSchema.setSchemaId(long versionId) Set the version ID for this schema.

Methods in org.apache.hudi.internal.schema with parameters of type InternalSchema

Modifier and Type	Method and Description
<T> T	InternalSchemaBuilder.visit(InternalSchema schema, InternalSchemaVisitor<T> visitor) Use to traverse all types in internalSchema with visitor.

Uses of InternalSchema in org.apache.hudi.internal.schema.action

Fields in org.apache.hudi.internal.schema.action declared as InternalSchema

Modifier and Type	Field and Description
protected InternalSchema	TableChange.BaseColumnChange.internalSchema

Methods in org.apache.hudi.internal.schema.action that return InternalSchema

Modifier and Type	Method and Description
InternalSchema	InternalSchemaChangeApplier.applyAddChange(String colName, Type colType, String doc, String position, TableChange.ColumnPositionChange.ColumnPositionType positionType) Add columns to table.
InternalSchema	InternalSchemaChangeApplier.applyColumnCommentChange(String colName, String doc) Update col comment for hudi table.
InternalSchema	InternalSchemaChangeApplier.applyColumnNullabilityChange(String colName, boolean nullable) Update col nullability for hudi table.
InternalSchema	InternalSchemaChangeApplier.applyColumnTypeChange(String colName, Type newType) Update col type for hudi table.
InternalSchema	InternalSchemaChangeApplier.applyDeleteChange(String... colNames) Delete columns to table.
InternalSchema	InternalSchemaChangeApplier.applyRenameChange(String colName, String newName) Rename col name for hudi table.
InternalSchema	InternalSchemaChangeApplier.applyReOrderColPositionChange(String colName, String referColName, TableChange.ColumnPositionChange.ColumnPositionType positionType) Reorder the position of col.
InternalSchema	InternalSchemaMerger.mergeSchema() Create final read schema to read avro/parquet file.

Methods in org.apache.hudi.internal.schema.action that return types with arguments of type InternalSchema

Modifier and Type	Method and Description
Pair<InternalSchema,Map<String,String>>	InternalSchemaMerger.mergeSchemaGetRenamed() Create final read schema to read avro/parquet file.

Methods in org.apache.hudi.internal.schema.action with parameters of type InternalSchema

Modifier and Type	Method and Description
static TableChanges.ColumnUpdateChange	TableChanges.ColumnUpdateChange.get(InternalSchema schema)
static TableChanges.ColumnDeleteChange	TableChanges.ColumnDeleteChange.get(InternalSchema schema)
static TableChanges.ColumnAddChange	TableChanges.ColumnAddChange.get(InternalSchema internalSchema)
static TableChanges.ColumnUpdateChange	TableChanges.ColumnUpdateChange.get(InternalSchema schema, boolean caseSensitive)

Constructors in org.apache.hudi.internal.schema.action with parameters of type InternalSchema

Constructor and Description
InternalSchemaChangeApplier(InternalSchema latestSchema)
InternalSchemaMerger(InternalSchema fileSchema, InternalSchema querySchema, boolean ignoreRequiredAttribute, boolean useColumnTypeFromFileSchema)
InternalSchemaMerger(InternalSchema fileSchema, InternalSchema querySchema, boolean ignoreRequiredAttribute, boolean useColumnTypeFromFileSchema, boolean useColNameFromFileSchema)

Uses of InternalSchema in org.apache.hudi.internal.schema.convert

Methods in org.apache.hudi.internal.schema.convert that return InternalSchema

Modifier and Type	Method and Description
static InternalSchema	AvroInternalSchemaConverter.convert(org.apache.avro.Schema schema)
static InternalSchema	AvroInternalSchemaConverter.convert(org.apache.avro.Schema schema, Map<String,Integer> existingFieldNameToPositionMapping) Convert an avro schema into internalSchema.
static InternalSchema	AvroInternalSchemaConverter.pruneAvroSchemaToInternalSchema(org.apache.avro.Schema schema, InternalSchema originSchema)

Methods in org.apache.hudi.internal.schema.convert with parameters of type InternalSchema

Modifier and Type	Method and Description
static org.apache.avro.Schema	AvroInternalSchemaConverter.buildAvroSchemaFromInternalSchema(InternalSchema schema, String recordName) Converts hudi internal Schema into an Avro Schema.
static org.apache.avro.Schema	AvroInternalSchemaConverter.convert(InternalSchema internalSchema, String name) Convert internalSchema to avro Schema.
static InternalSchema	AvroInternalSchemaConverter.pruneAvroSchemaToInternalSchema(org.apache.avro.Schema schema, InternalSchema originSchema)

Uses of InternalSchema in org.apache.hudi.internal.schema.io

Methods in org.apache.hudi.internal.schema.io that return types with arguments of type InternalSchema

Modifier and Type	Method and Description
Option<InternalSchema>	FileBasedInternalSchemaStorageManager.getSchemaByKey(String versionId)

Uses of InternalSchema in org.apache.hudi.internal.schema.utils

Methods in org.apache.hudi.internal.schema.utils that return InternalSchema

Modifier and Type	Method and Description
static InternalSchema	SchemaChangeUtils.applyTableChanges2Schema(InternalSchema internalSchema, TableChanges.ColumnAddChange adds) Apply all the DDL add operations to internalSchema to produce a new internalSchema.
static InternalSchema	SchemaChangeUtils.applyTableChanges2Schema(InternalSchema internalSchema, TableChanges.ColumnDeleteChange deletes) Apply all the DDL delete operations to internalSchema to produce a new internalSchema.
static InternalSchema	SchemaChangeUtils.applyTableChanges2Schema(InternalSchema internalSchema, TableChanges.ColumnUpdateChange updates) Apply all the DDL update operations to internalSchema to produce a new internalSchema.
static InternalSchema	SerDeHelper.fromJson(com.fasterxml.jackson.databind.JsonNode jsonNode) Convert jsonNode to internalSchema.
static InternalSchema	InternalSchemaUtils.pruneInternalSchema(InternalSchema schema, List<String> names) Create project internalSchema, based on the project names which produced by query engine.
static InternalSchema	InternalSchemaUtils.pruneInternalSchemaByID(InternalSchema schema, List<Integer> fieldIds, List<Integer> topParentFieldIds) Create project internalSchema.
static InternalSchema	AvroSchemaEvolutionUtils.reconcileSchema(org.apache.avro.Schema incomingSchema, InternalSchema oldTableSchema, boolean makeMissingFieldsNullable) Support reconcile from a new avroSchema.
static InternalSchema	InternalSchemaUtils.searchSchema(long versionId, List<InternalSchema> internalSchemas) Search target internalSchema by version number.
static InternalSchema	InternalSchemaUtils.searchSchema(long versionId, TreeMap<Long,InternalSchema> treeMap) Search target internalSchema by version number.

Methods in org.apache.hudi.internal.schema.utils that return types with arguments of type InternalSchema

Modifier and Type	Method and Description
static Option<InternalSchema>	SerDeHelper.fromJson(String json) Convert string to internalSchema.
static TreeMap<Long,InternalSchema>	SerDeHelper.parseSchemas(String json) Convert json string to history internalSchemas.

Methods in org.apache.hudi.internal.schema.utils with parameters of type InternalSchema

Modifier and Type	Method and Description
static InternalSchema	SchemaChangeUtils.applyTableChanges2Schema(InternalSchema internalSchema, TableChanges.ColumnAddChange adds) Apply all the DDL add operations to internalSchema to produce a new internalSchema.
static InternalSchema	SchemaChangeUtils.applyTableChanges2Schema(InternalSchema internalSchema, TableChanges.ColumnDeleteChange deletes) Apply all the DDL delete operations to internalSchema to produce a new internalSchema.
static InternalSchema	SchemaChangeUtils.applyTableChanges2Schema(InternalSchema internalSchema, TableChanges.ColumnUpdateChange updates) Apply all the DDL update operations to internalSchema to produce a new internalSchema.
static Map<String,String>	InternalSchemaUtils.collectRenameCols(InternalSchema oldSchema, InternalSchema newSchema) Try to find all renamed cols between oldSchema and newSchema.
static Map<Integer,Pair<Type,Type>>	InternalSchemaUtils.collectTypeChangedCols(InternalSchema schema, InternalSchema oldSchema) Collect all type changed cols to build a colPosition -> (newColType, oldColType) map.
static String	SerDeHelper.inheritSchemas(InternalSchema newSchema, String oldSchemas) Add the new schema to the historical schemas.
static InternalSchema	InternalSchemaUtils.pruneInternalSchema(InternalSchema schema, List<String> names) Create project internalSchema, based on the project names which produced by query engine.
static InternalSchema	InternalSchemaUtils.pruneInternalSchemaByID(InternalSchema schema, List<Integer> fieldIds, List<Integer> topParentFieldIds) Create project internalSchema.
static String	InternalSchemaUtils.reBuildFilterName(String name, InternalSchema fileSchema, InternalSchema querySchema) A helper function to help correct the colName of pushed filters.
static InternalSchema	AvroSchemaEvolutionUtils.reconcileSchema(org.apache.avro.Schema incomingSchema, InternalSchema oldTableSchema, boolean makeMissingFieldsNullable) Support reconcile from a new avroSchema.
static String	SerDeHelper.toJson(InternalSchema internalSchema) Convert internalSchemas to json.

Method parameters in org.apache.hudi.internal.schema.utils with type arguments of type InternalSchema

Modifier and Type	Method and Description
static InternalSchema	InternalSchemaUtils.searchSchema(long versionId, List<InternalSchema> internalSchemas) Search target internalSchema by version number.
static InternalSchema	InternalSchemaUtils.searchSchema(long versionId, TreeMap<Long,InternalSchema> treeMap) Search target internalSchema by version number.
static String	SerDeHelper.toJson(List<InternalSchema> internalSchemas) Convert history internalSchemas to json.

Uses of InternalSchema in org.apache.hudi.internal.schema.visitor

Methods in org.apache.hudi.internal.schema.visitor with parameters of type InternalSchema

Modifier and Type	Method and Description
Map<String,Integer>	NameToPositionVisitor.schema(InternalSchema schema, Map<String,Integer> recordResult)
Map<String,Integer>	NameToIDVisitor.schema(InternalSchema schema, Map<String,Integer> recordResult)
T	InternalSchemaVisitor.schema(InternalSchema schema, T recordResult)