org.apache.hadoop.hbase.io.hfile.bucket

Class BucketCache

java.lang.Object

org.apache.hadoop.hbase.io.hfile.bucket.BucketCache

All Implemented Interfaces:

HeapSize, BlockCache

@InterfaceAudience.Private
public class BucketCache
extends Object
implements BlockCache, HeapSize

BucketCache uses BucketAllocator to allocate/free block, and use ramCache and backingMap in order to determine whether a given element hit. It could uses memory ByteBufferIOEngine or file FileIOEngineto store/read the block data. Eviction is using similar algorithm as LruBlockCache BucketCache could be used as mainly a block cache(see CombinedBlockCache), combined with LruBlockCache to decrease CMS and fragment by GC. Also could be used as a secondary cache(e.g. using Fusionio to store block) to enlarge cache space by LruBlockCache.setVictimCache(org.apache.hadoop.hbase.io.hfile.bucket.BucketCache)

Field Summary

Fields

Modifier and Type	Field and Description
static int	DEFAULT_ERROR_TOLERATION_DURATION

Constructor Summary

Constructors

Constructor and Description
BucketCache(String ioEngineName, long capacity, int writerThreadNum, int writerQLen, String persistencePath)
BucketCache(String ioEngineName, long capacity, int writerThreadNum, int writerQLen, String persistencePath, int ioErrorsTolerationDuration)

Method Summary

Methods

Modifier and Type	Method and Description
void	cacheBlock(BlockCacheKey cacheKey, Cacheable buf) Cache the block with the specified name and buffer.
void	cacheBlock(BlockCacheKey cacheKey, Cacheable cachedItem, boolean inMemory) Cache the block with the specified name and buffer.
void	cacheBlockWithWait(BlockCacheKey cacheKey, Cacheable cachedItem, boolean inMemory, boolean wait) Cache the block to ramCache
boolean	evictBlock(BlockCacheKey cacheKey) Evict block from cache.
int	evictBlocksByHfileName(String hfileName) Evicts all blocks for a specific HFile.
Cacheable	getBlock(BlockCacheKey key, boolean caching, boolean repeat) Get the buffer of the block with the specified key.
List<BlockCacheColumnFamilySummary>	getBlockCacheColumnFamilySummaries(org.apache.hadoop.conf.Configuration conf) Performs a BlockCache summary and returns a List of BlockCacheColumnFamilySummary objects.
long	getBlockCount() Returns the number of blocks currently cached in the block cache.
long	getCurrentSize() Returns the occupied size of the block cache, in bytes.
long	getEvictedCount() Returns the number of blocks that have been evicted.
long	getFreeSize() Returns the free size of the block cache, in bytes.
CacheStats	getStats() Get the statistics for this block cache.
long	heapSize()
void	logStats()
void	shutdown() Shutdown the cache.
long	size() Returns the total size of the block cache, in bytes.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DEFAULT_ERROR_TOLERATION_DURATION

public static final int DEFAULT_ERROR_TOLERATION_DURATION

See Also:

Constant Field Values

Constructor Detail

BucketCache

public BucketCache(String ioEngineName,
           long capacity,
           int writerThreadNum,
           int writerQLen,
           String persistencePath)
            throws FileNotFoundException,
                   IOException

Throws:

FileNotFoundException

IOException

BucketCache

public BucketCache(String ioEngineName,
           long capacity,
           int writerThreadNum,
           int writerQLen,
           String persistencePath,
           int ioErrorsTolerationDuration)
            throws FileNotFoundException,
                   IOException

Throws:

FileNotFoundException

IOException

Method Detail

cacheBlock

public void cacheBlock(BlockCacheKey cacheKey,
              Cacheable buf)

Cache the block with the specified name and buffer.

Specified by:

cacheBlock in interface BlockCache

Parameters:

cacheKey - block's cache key

buf - block buffer

cacheBlock

public void cacheBlock(BlockCacheKey cacheKey,
              Cacheable cachedItem,
              boolean inMemory)

Cache the block with the specified name and buffer.

Specified by:

cacheBlock in interface BlockCache

Parameters:

cacheKey - block's cache key

cachedItem - block buffer

inMemory - if block is in-memory

cacheBlockWithWait

public void cacheBlockWithWait(BlockCacheKey cacheKey,
                      Cacheable cachedItem,
                      boolean inMemory,
                      boolean wait)

Cache the block to ramCache

Parameters:

cacheKey - block's cache key

cachedItem - block buffer

inMemory - if block is in-memory

wait - if true, blocking wait when queue is full

getBlock

public Cacheable getBlock(BlockCacheKey key,
                 boolean caching,
                 boolean repeat)

Get the buffer of the block with the specified key.

Specified by:

getBlock in interface BlockCache

Parameters:

key - block's cache key

caching - true if the caller caches blocks on cache misses

repeat - Whether this is a repeat lookup for the same block

Returns:

buffer of specified cache key, or null if not in cache

See Also:

HFileReaderV2.readBlock(long, long, boolean, boolean, boolean, BlockType)

evictBlock

public boolean evictBlock(BlockCacheKey cacheKey)

Description copied from interface: BlockCache

Evict block from cache.

Specified by:

evictBlock in interface BlockCache

Parameters:

cacheKey - Block to evict

Returns:

true if block existed and was evicted, false if not

logStats

public void logStats()

shutdown

public void shutdown()

Description copied from interface: BlockCache

Shutdown the cache.

Specified by:

shutdown in interface BlockCache

getStats

public CacheStats getStats()

Description copied from interface: BlockCache

Get the statistics for this block cache.

Specified by:

getStats in interface BlockCache

Returns:

Stats

heapSize

public long heapSize()

Specified by:

heapSize in interface HeapSize

size

public long size()

Description copied from interface: BlockCache

Returns the total size of the block cache, in bytes.

Specified by:

size in interface BlockCache

Returns:

size of cache, in bytes

getFreeSize

public long getFreeSize()

Description copied from interface: BlockCache

Returns the free size of the block cache, in bytes.

Specified by:

getFreeSize in interface BlockCache

Returns:

free space in cache, in bytes

getBlockCount

public long getBlockCount()

Description copied from interface: BlockCache

Returns the number of blocks currently cached in the block cache.

Specified by:

getBlockCount in interface BlockCache

Returns:

number of blocks in the cache

getCurrentSize

public long getCurrentSize()

Description copied from interface: BlockCache

Returns the occupied size of the block cache, in bytes.

Specified by:

getCurrentSize in interface BlockCache

Returns:

occupied space in cache, in bytes

getEvictedCount

public long getEvictedCount()

Description copied from interface: BlockCache

Returns the number of blocks that have been evicted.

Specified by:

getEvictedCount in interface BlockCache

Returns:

number of evicted blocks

evictBlocksByHfileName

public int evictBlocksByHfileName(String hfileName)

Evicts all blocks for a specific HFile.

This is used for evict-on-close to remove all blocks of a specific HFile.

Specified by:

evictBlocksByHfileName in interface BlockCache

Returns:

the number of blocks evicted

getBlockCacheColumnFamilySummaries

public List<BlockCacheColumnFamilySummary> getBlockCacheColumnFamilySummaries(org.apache.hadoop.conf.Configuration conf)

Description copied from interface: BlockCache

Performs a BlockCache summary and returns a List of BlockCacheColumnFamilySummary objects. This method could be fairly heavyweight in that it evaluates the entire HBase file-system against what is in the RegionServer BlockCache.

The contract of this interface is to return the List in sorted order by Table name, then ColumnFamily.

Specified by:

getBlockCacheColumnFamilySummaries in interface BlockCache

Parameters:

conf - HBaseConfiguration

Returns:

List of BlockCacheColumnFamilySummary