org.apache.hudi.index.bucket

Class HoodieConsistentBucketIndex

java.lang.Object

org.apache.hudi.index.HoodieIndex<Object,Object>

org.apache.hudi.index.bucket.HoodieBucketIndex

org.apache.hudi.index.bucket.HoodieConsistentBucketIndex

All Implemented Interfaces:

Serializable

public class HoodieConsistentBucketIndex
extends HoodieBucketIndex

Consistent hashing bucket index implementation, with auto-adjust bucket number. NOTE: bucket resizing is triggered by clustering.

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
class	HoodieConsistentBucketIndex.ConsistentBucketIndexLocationMapper

Nested classes/interfaces inherited from class org.apache.hudi.index.HoodieIndex

HoodieIndex.BucketIndexEngineType, HoodieIndex.IndexType

Field Summary

Fields inherited from class org.apache.hudi.index.bucket.HoodieBucketIndex

indexKeyFields, numBuckets

Fields inherited from class org.apache.hudi.index.HoodieIndex

config

Constructor Summary

Constructors

Constructor and Description
HoodieConsistentBucketIndex(HoodieWriteConfig config)

Method Summary

Modifier and Type	Method and Description
boolean	rollbackCommit(String instantTime) Do nothing.
<R> HoodieData<HoodieRecord<R>>	tagLocation(HoodieData<HoodieRecord<R>> records, HoodieEngineContext context, HoodieTable hoodieTable) Looks up the index and tags each incoming record with a location of a file that contains the row (if it is actually present).
HoodieData<WriteStatus>	updateLocation(HoodieData<WriteStatus> writeStatuses, HoodieEngineContext context, HoodieTable hoodieTable) Extracts the location of written records, and updates the index.

Methods inherited from class org.apache.hudi.index.bucket.HoodieBucketIndex

canIndexLogFiles, getNumBuckets, isGlobal, isImplicitWithStorage, requiresTagging

Methods inherited from class org.apache.hudi.index.HoodieIndex

close, tagLocation, updateLocation, updateLocation

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HoodieConsistentBucketIndex

public HoodieConsistentBucketIndex(HoodieWriteConfig config)

Method Detail

updateLocation

public HoodieData<WriteStatus> updateLocation(HoodieData<WriteStatus> writeStatuses,
                                              HoodieEngineContext context,
                                              HoodieTable hoodieTable)
                                       throws HoodieIndexException

Description copied from class: HoodieIndex

Extracts the location of written records, and updates the index.

Overrides:

updateLocation in class HoodieBucketIndex

Throws:

HoodieIndexException

rollbackCommit

public boolean rollbackCommit(String instantTime)

Do nothing. A failed write may create a hashing metadata for a partition. In this case, we still do nothing when rolling back the failed write. Because the hashing metadata created by a writer must have 00000000000000 timestamp and can be viewed as the initialization of a partition rather than as a part of the failed write.

Overrides:

rollbackCommit in class HoodieBucketIndex

tagLocation

public <R> HoodieData<HoodieRecord<R>> tagLocation(HoodieData<HoodieRecord<R>> records,
                                                   HoodieEngineContext context,
                                                   HoodieTable hoodieTable)
                                            throws HoodieIndexException

Description copied from class: HoodieIndex

Looks up the index and tags each incoming record with a location of a file that contains the row (if it is actually present).

Specified by:

tagLocation in class HoodieIndex<Object,Object>

Throws:

HoodieIndexException