org.apache.hadoop.hbase.util

Class CompoundBloomFilterWriter

java.lang.Object

org.apache.hadoop.hbase.util.CompoundBloomFilterBase

org.apache.hadoop.hbase.util.CompoundBloomFilterWriter

All Implemented Interfaces:

InlineBlockWriter, BloomFilterBase, BloomFilterWriter

@InterfaceAudience.Private
public class CompoundBloomFilterWriter
extends CompoundBloomFilterBase
implements BloomFilterWriter, InlineBlockWriter

Adds methods required for writing a compound Bloom filter to the data section of an HFile to the CompoundBloomFilter class.

Field Summary

Fields

Modifier and Type	Field and Description
protected static org.apache.commons.logging.Log	LOG

Fields inherited from class org.apache.hadoop.hbase.util.CompoundBloomFilterBase

comparator, errorRate, hashType, numChunks, totalByteSize, totalKeyCount, totalMaxKeys, VERSION

Constructor Summary

Constructors

Constructor and Description
CompoundBloomFilterWriter(int chunkByteSizeHint, float errorRate, int hashType, int maxFold, boolean cacheOnWrite, KeyValue.KVComparator comparator)

Method Summary

Methods

Modifier and Type	Method and Description
void	add(byte[] bloomKey, int keyOffset, int keyLength) Adds a Bloom filter key.
void	allocBloom() Allocate memory for the bloom filter data.
void	blockWritten(long offset, int onDiskSize, int uncompressedSize) Called after a block has been written, and its offset, raw size, and compressed size have been determined.
void	compactBloom() Compact the Bloom filter before writing metadata & data to disk.
boolean	getCacheOnWrite()
org.apache.hadoop.io.Writable	getDataWriter() Get a writable interface into bloom filter data (the actual Bloom bits).
BlockType	getInlineBlockType() The type of blocks this block writer produces.
org.apache.hadoop.io.Writable	getMetaWriter() Get a writable interface into bloom filter meta data.
boolean	shouldWriteBlock(boolean closing) Determines whether there is a new block to be written out.
void	writeInlineBlock(DataOutput out) Writes the block to the provided stream.

Methods inherited from class org.apache.hadoop.hbase.util.CompoundBloomFilterBase

createBloomKey, getByteSize, getComparator, getKeyCount, getMaxKeys

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.hbase.util.BloomFilterBase

createBloomKey, getByteSize, getComparator, getKeyCount, getMaxKeys

Field Detail

LOG

protected static final org.apache.commons.logging.Log LOG

Constructor Detail

CompoundBloomFilterWriter

public CompoundBloomFilterWriter(int chunkByteSizeHint,
                         float errorRate,
                         int hashType,
                         int maxFold,
                         boolean cacheOnWrite,
                         KeyValue.KVComparator comparator)

Parameters:

chunkByteSizeHint - each chunk's size in bytes. The real chunk size might be different as required by the fold factor.

errorRate - target false positive rate

hashType - hash function type to use

maxFold - maximum degree of folding allowed

Method Detail

shouldWriteBlock

public boolean shouldWriteBlock(boolean closing)

Description copied from interface: InlineBlockWriter

Determines whether there is a new block to be written out.

Specified by:

shouldWriteBlock in interface InlineBlockWriter

Parameters:

closing - whether the file is being closed, in which case we need to write out all available data and not wait to accumulate another block

add

public void add(byte[] bloomKey,
       int keyOffset,
       int keyLength)

Adds a Bloom filter key. This key must be greater than the previous key, as defined by the comparator this compound Bloom filter is configured with. For efficiency, key monotonicity is not checked here. See StoreFile.Writer.append( org.apache.hadoop.hbase.KeyValue) for the details of deduplication.

Specified by:

add in interface BloomFilterWriter

Parameters:

bloomKey - data to be added to the bloom

keyOffset - offset into the data to be added

keyLength - length of the data to be added

writeInlineBlock

public void writeInlineBlock(DataOutput out)
                      throws IOException

Description copied from interface: InlineBlockWriter

Writes the block to the provided stream. Must not write any magic records. Called only if InlineBlockWriter.shouldWriteBlock(boolean) returned true.

Specified by:

writeInlineBlock in interface InlineBlockWriter

Parameters:

out - a stream (usually a compressing stream) to write the block to

Throws:

IOException

blockWritten

public void blockWritten(long offset,
                int onDiskSize,
                int uncompressedSize)

Description copied from interface: InlineBlockWriter

Called after a block has been written, and its offset, raw size, and compressed size have been determined. Can be used to add an entry to a block index. If this type of inline blocks needs a block index, the inline block writer is responsible for maintaining it.

Specified by:

blockWritten in interface InlineBlockWriter

Parameters:

offset - the offset of the block in the stream

onDiskSize - the on-disk size of the block

uncompressedSize - the uncompressed size of the block

getInlineBlockType

public BlockType getInlineBlockType()

Description copied from interface: InlineBlockWriter

The type of blocks this block writer produces.

Specified by:

getInlineBlockType in interface InlineBlockWriter

getMetaWriter

public org.apache.hadoop.io.Writable getMetaWriter()

Description copied from interface: BloomFilterWriter

Get a writable interface into bloom filter meta data.

Specified by:

getMetaWriter in interface BloomFilterWriter

Returns:

a writable instance that can be later written to a stream

compactBloom

public void compactBloom()

Description copied from interface: BloomFilterWriter

Compact the Bloom filter before writing metadata & data to disk.

Specified by:

compactBloom in interface BloomFilterWriter

allocBloom

public void allocBloom()

Description copied from interface: BloomFilterWriter

Allocate memory for the bloom filter data.

Specified by:

allocBloom in interface BloomFilterWriter

getDataWriter

public org.apache.hadoop.io.Writable getDataWriter()

Description copied from interface: BloomFilterWriter

Get a writable interface into bloom filter data (the actual Bloom bits). Not used for compound Bloom filters.

Specified by:

getDataWriter in interface BloomFilterWriter

Returns:

a writable instance that can be later written to a stream

getCacheOnWrite

public boolean getCacheOnWrite()

Specified by:

getCacheOnWrite in interface InlineBlockWriter

Returns:

true if inline blocks produced by this writer should be cached