org.apache.hudi.index

Class HoodieIndex<I,O>

java.lang.Object

org.apache.hudi.index.HoodieIndex<I,O>

Type Parameters:

I - Type of inputs for deprecated APIs

O - Type of outputs for deprecated APIs

All Implemented Interfaces:

Serializable

Direct Known Subclasses:

HoodieBloomIndex, HoodieBucketIndex, HoodieInMemoryHashIndex, HoodieInternalProxyIndex, HoodieSimpleIndex

@PublicAPIClass(maturity=EVOLVING)
public abstract class HoodieIndex<I,O>
extends Object
implements Serializable

Base class for different types of indexes to determine the mapping from uuid.

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	HoodieIndex.BucketIndexEngineType
static class	HoodieIndex.IndexType

Field Summary

Fields

Modifier and Type	Field and Description
protected HoodieWriteConfig	config

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	HoodieIndex(HoodieWriteConfig config)

Method Summary

Modifier and Type	Method and Description
abstract boolean	canIndexLogFiles() This is used by storage to determine, if it is safe to send inserts, straight to the log, i.e.
void	close() Each index type should implement its own logic to release any resources acquired during the process.
abstract boolean	isGlobal() An index is `global` if HoodieKey to fileID mapping, does not depend on the `partitionPath`.
abstract boolean	isImplicitWithStorage() An index is "implicit" with respect to storage, if just writing new data to a file slice, updates the index as well.
boolean	requiresTagging(WriteOperationType operationType) To indicate if an operation type requires location tagging before writing
abstract boolean	rollbackCommit(String instantTime) Rollback the effects of the commit made at instantTime.
abstract <R> HoodieData<HoodieRecord<R>>	tagLocation(HoodieData<HoodieRecord<R>> records, HoodieEngineContext context, HoodieTable hoodieTable) Looks up the index and tags each incoming record with a location of a file that contains the row (if it is actually present).
I	tagLocation(I records, HoodieEngineContext context, HoodieTable hoodieTable) Deprecated.
abstract HoodieData<WriteStatus>	updateLocation(HoodieData<WriteStatus> writeStatuses, HoodieEngineContext context, HoodieTable hoodieTable) Extracts the location of written records, and updates the index.
HoodieData<WriteStatus>	updateLocation(HoodieData<WriteStatus> writeStatuses, HoodieEngineContext context, HoodieTable hoodieTable, String instant) Extracts the location of written records, and updates the index.
O	updateLocation(O writeStatuses, HoodieEngineContext context, HoodieTable hoodieTable) Deprecated.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

config

protected final HoodieWriteConfig config

Constructor Detail

HoodieIndex

protected HoodieIndex(HoodieWriteConfig config)

Method Detail

tagLocation

@Deprecated
 @PublicAPIMethod(maturity=DEPRECATED)
public I tagLocation(I records,
                                                                        HoodieEngineContext context,
                                                                        HoodieTable hoodieTable)
                                                                 throws HoodieIndexException

Deprecated.

Looks up the index and tags each incoming record with a location of a file that contains the row (if it is actually present).

Throws:

HoodieIndexException

updateLocation

@Deprecated
 @PublicAPIMethod(maturity=DEPRECATED)
public O updateLocation(O writeStatuses,
                                                                           HoodieEngineContext context,
                                                                           HoodieTable hoodieTable)
                                                                    throws HoodieIndexException

Deprecated.

Extracts the location of written records, and updates the index.

Throws:

HoodieIndexException

tagLocation

@PublicAPIMethod(maturity=EVOLVING)
public abstract <R> HoodieData<HoodieRecord<R>> tagLocation(HoodieData<HoodieRecord<R>> records,
                                                                                                HoodieEngineContext context,
                                                                                                HoodieTable hoodieTable)
                                                                                         throws HoodieIndexException

Looks up the index and tags each incoming record with a location of a file that contains the row (if it is actually present).

Throws:

HoodieIndexException

updateLocation

@PublicAPIMethod(maturity=EVOLVING)
public abstract HoodieData<WriteStatus> updateLocation(HoodieData<WriteStatus> writeStatuses,
                                                                                           HoodieEngineContext context,
                                                                                           HoodieTable hoodieTable)
                                                                                    throws HoodieIndexException

Extracts the location of written records, and updates the index.

Throws:

HoodieIndexException

updateLocation

@PublicAPIMethod(maturity=EVOLVING)
public HoodieData<WriteStatus> updateLocation(HoodieData<WriteStatus> writeStatuses,
                                                                                  HoodieEngineContext context,
                                                                                  HoodieTable hoodieTable,
                                                                                  String instant)
                                                                           throws HoodieIndexException

Extracts the location of written records, and updates the index.

Throws:

HoodieIndexException

rollbackCommit

@PublicAPIMethod(maturity=STABLE)
public abstract boolean rollbackCommit(String instantTime)

Rollback the effects of the commit made at instantTime.

isGlobal

@PublicAPIMethod(maturity=STABLE)
public abstract boolean isGlobal()

An index is `global` if HoodieKey to fileID mapping, does not depend on the `partitionPath`. Such an implementation is able to obtain the same mapping, for two hoodie keys with same `recordKey` but different `partitionPath`

Returns:

whether the index implementation is global in nature

canIndexLogFiles

@PublicAPIMethod(maturity=EVOLVING)
public abstract boolean canIndexLogFiles()

This is used by storage to determine, if it is safe to send inserts, straight to the log, i.e. having a FileSlice, with no data file.

Returns:

Returns true/false depending on whether the impl has this capability

isImplicitWithStorage

@PublicAPIMethod(maturity=STABLE)
public abstract boolean isImplicitWithStorage()

An index is "implicit" with respect to storage, if just writing new data to a file slice, updates the index as well. This is used by storage, to save memory footprint in certain cases.

requiresTagging

@PublicAPIMethod(maturity=EVOLVING)
public boolean requiresTagging(WriteOperationType operationType)

To indicate if an operation type requires location tagging before writing

close

public void close()

Each index type should implement its own logic to release any resources acquired during the process.