org.apache.hadoop.hbase.regionserver

Class HStoreFile

java.lang.Object

org.apache.hadoop.hbase.regionserver.HStoreFile

All Implemented Interfaces:

StoreFile

@InterfaceAudience.Private
public class HStoreFile
extends Object
implements StoreFile

A Store data file. Stores usually have one or more of these files. They are produced by flushing the memstore to disk. To create, instantiate a writer using StoreFileWriter.Builder and append data. Be sure to add any metadata before calling close on the Writer (Use the appendMetadata convenience methods). On close, a StoreFile is sitting in the Filesystem. To refer to it, create a StoreFile instance passing filesystem and path. To read, call initReader()

StoreFiles may also reference store files in another Store. The reason for this weird pattern where you use a different instance for the writer and a reader is that we write once but read a lot more.

Field Summary

Fields inherited from interface org.apache.hadoop.hbase.regionserver.StoreFile

BLOOM_FILTER_TYPE_KEY, BULKLOAD_TASK_KEY, BULKLOAD_TIME_KEY, DELETE_FAMILY_COUNT, EARLIEST_PUT_TS, EXCLUDE_FROM_MINOR_COMPACTION_KEY, LAST_BLOOM_KEY, MAJOR_COMPACTION_KEY, MAX_SEQ_ID_KEY, MOB_CELLS_COUNT, SKIP_RESET_SEQ_ID, STORE_FILE_READER_NO_READAHEAD, TIMERANGE_KEY

Constructor Summary

Constructors

Constructor and Description
HStoreFile(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path p, org.apache.hadoop.conf.Configuration conf, CacheConfig cacheConf, BloomType cfBloomType) Deprecated. Now we will specific whether the StoreFile is for primary replica when constructing, so please use HStoreFile(FileSystem, Path, Configuration, CacheConfig, BloomType, boolean) directly.
HStoreFile(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path p, org.apache.hadoop.conf.Configuration conf, CacheConfig cacheConf, BloomType cfBloomType, boolean primaryReplica) Constructor, loads a reader and it's indices, etc.
HStoreFile(org.apache.hadoop.fs.FileSystem fs, StoreFileInfo fileInfo, org.apache.hadoop.conf.Configuration conf, CacheConfig cacheConf, BloomType cfBloomType) Deprecated. Now we will specific whether the StoreFile is for primary replica when constructing, so please use HStoreFile(FileSystem, StoreFileInfo, Configuration, CacheConfig, BloomType, boolean) directly.
HStoreFile(org.apache.hadoop.fs.FileSystem fs, StoreFileInfo fileInfo, org.apache.hadoop.conf.Configuration conf, CacheConfig cacheConf, BloomType cfBloomType, boolean primaryReplica) Constructor, loads a reader and it's indices, etc.

Method Summary

Modifier and Type	Method and Description
void	closeReader(boolean evictOnClose)
void	deleteReader() Delete this file
boolean	excludeFromMinorCompaction()
OptionalLong	getBulkLoadTimestamp() Return the timestamp at which this bulk load file was generated.
CacheConfig	getCacheConf()
Comparator<Cell>	getComparator()
StoreFileInfo	getFileInfo()
Cell	getFirstKey()
HDFSBlocksDistribution	getHDFSBlockDistribution()
Cell	getLastKey()
OptionalLong	getMaximumTimestamp()
long	getMaxMemstoreTS()
long	getMaxSequenceId()
byte[]	getMetadataValue(byte[] key) Only used by the Striped Compaction Policy
OptionalLong	getMinimumTimestamp()
long	getModificationTimeStamp()
org.apache.hadoop.fs.Path	getPath()
StoreFileScanner	getPreadScanner(boolean cacheBlocks, long readPt, long scannerOrder, boolean canOptimizeForNonNullColumn) Must be called after initReader.
org.apache.hadoop.fs.Path	getQualifiedPath()
StoreFileReader	getReader()
int	getRefCount()
StoreFileScanner	getStreamScanner(boolean canUseDropBehind, boolean cacheBlocks, boolean isCompaction, long readPt, long scannerOrder, boolean canOptimizeForNonNullColumn)
void	initReader() Initialize the reader used for pread.
boolean	isBulkLoadResult() Check if this storefile was created by bulk load.
boolean	isCompactedAway()
boolean	isHFile()
boolean	isMajorCompactionResult()
boolean	isReference()
boolean	isReferencedInReads()
void	markCompactedAway() Marks the status of the file as compactedAway.
String	toString()
String	toStringDetailed()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

HStoreFile

@Deprecated
public HStoreFile(org.apache.hadoop.fs.FileSystem fs,
                              org.apache.hadoop.fs.Path p,
                              org.apache.hadoop.conf.Configuration conf,
                              CacheConfig cacheConf,
                              BloomType cfBloomType)
                       throws IOException

Deprecated. Now we will specific whether the StoreFile is for primary replica when constructing, so please use HStoreFile(FileSystem, Path, Configuration, CacheConfig, BloomType, boolean) directly.

Constructor, loads a reader and it's indices, etc. May allocate a substantial amount of ram depending on the underlying files (10-20MB?).

Parameters:

fs - The current file system to use.

p - The path of the file.

conf - The current configuration.

cacheConf - The cache configuration and block cache reference.

cfBloomType - The bloom type to use for this store file as specified by column family configuration. This may or may not be the same as the Bloom filter type actually present in the HFile, because column family configuration might change. If this is BloomType.NONE, the existing Bloom filter is ignored.

Throws:

IOException

HStoreFile

public HStoreFile(org.apache.hadoop.fs.FileSystem fs,
                  org.apache.hadoop.fs.Path p,
                  org.apache.hadoop.conf.Configuration conf,
                  CacheConfig cacheConf,
                  BloomType cfBloomType,
                  boolean primaryReplica)
           throws IOException

Constructor, loads a reader and it's indices, etc. May allocate a substantial amount of ram depending on the underlying files (10-20MB?).

Parameters:

fs - The current file system to use.

p - The path of the file.

conf - The current configuration.

cacheConf - The cache configuration and block cache reference.

cfBloomType - The bloom type to use for this store file as specified by column family configuration. This may or may not be the same as the Bloom filter type actually present in the HFile, because column family configuration might change. If this is BloomType.NONE, the existing Bloom filter is ignored.

primaryReplica - true if this is a store file for primary replica, otherwise false.

Throws:

IOException

HStoreFile

@Deprecated
public HStoreFile(org.apache.hadoop.fs.FileSystem fs,
                              StoreFileInfo fileInfo,
                              org.apache.hadoop.conf.Configuration conf,
                              CacheConfig cacheConf,
                              BloomType cfBloomType)
                       throws IOException

Deprecated. Now we will specific whether the StoreFile is for primary replica when constructing, so please use HStoreFile(FileSystem, StoreFileInfo, Configuration, CacheConfig, BloomType, boolean) directly.

Constructor, loads a reader and it's indices, etc. May allocate a substantial amount of ram depending on the underlying files (10-20MB?).

Parameters:

fs - The current file system to use.

fileInfo - The store file information.

conf - The current configuration.

cacheConf - The cache configuration and block cache reference.

cfBloomType - The bloom type to use for this store file as specified by column family configuration. This may or may not be the same as the Bloom filter type actually present in the HFile, because column family configuration might change. If this is BloomType.NONE, the existing Bloom filter is ignored.

Throws:

IOException

HStoreFile

public HStoreFile(org.apache.hadoop.fs.FileSystem fs,
                  StoreFileInfo fileInfo,
                  org.apache.hadoop.conf.Configuration conf,
                  CacheConfig cacheConf,
                  BloomType cfBloomType,
                  boolean primaryReplica)

Constructor, loads a reader and it's indices, etc. May allocate a substantial amount of ram depending on the underlying files (10-20MB?).

Parameters:

fs - fs The current file system to use.

fileInfo - The store file information.

conf - The current configuration.

cacheConf - The cache configuration and block cache reference.

cfBloomType - The bloom type to use for this store file as specified by column family configuration. This may or may not be the same as the Bloom filter type actually present in the HFile, because column family configuration might change. If this is BloomType.NONE, the existing Bloom filter is ignored.

primaryReplica - true if this is a store file for primary replica, otherwise false.

Method Detail

getCacheConf

public CacheConfig getCacheConf()

Specified by:

getCacheConf in interface StoreFile

getFirstKey

public Cell getFirstKey()

Specified by:

getFirstKey in interface StoreFile

getLastKey

public Cell getLastKey()

Specified by:

getLastKey in interface StoreFile

getComparator

public Comparator<Cell> getComparator()

Specified by:

getComparator in interface StoreFile

getMaxMemstoreTS

public long getMaxMemstoreTS()

Specified by:

getMaxMemstoreTS in interface StoreFile

getFileInfo

public StoreFileInfo getFileInfo()

Specified by:

getFileInfo in interface StoreFile

Returns:

the StoreFile object associated to this StoreFile. null if the StoreFile is not a reference.

getPath

public org.apache.hadoop.fs.Path getPath()

Specified by:

getPath in interface StoreFile

Returns:

Path or null if this StoreFile was made with a Stream.

getQualifiedPath

public org.apache.hadoop.fs.Path getQualifiedPath()

Specified by:

getQualifiedPath in interface StoreFile

Returns:

Returns the qualified path of this StoreFile

isReference

public boolean isReference()

Specified by:

isReference in interface StoreFile

Returns:

True if this is a StoreFile Reference.

isHFile

public boolean isHFile()

Specified by:

isHFile in interface StoreFile

Returns:

True if this is HFile.

isMajorCompactionResult

public boolean isMajorCompactionResult()

Specified by:

isMajorCompactionResult in interface StoreFile

Returns:

True if this file was made by a major compaction.

excludeFromMinorCompaction

public boolean excludeFromMinorCompaction()

Specified by:

excludeFromMinorCompaction in interface StoreFile

Returns:

True if this file should not be part of a minor compaction.

getMaxSequenceId

public long getMaxSequenceId()

Specified by:

getMaxSequenceId in interface StoreFile

Returns:

This files maximum edit sequence id.

getModificationTimeStamp

public long getModificationTimeStamp()
                              throws IOException

Specified by:

getModificationTimeStamp in interface StoreFile

Throws:

IOException

getMetadataValue

public byte[] getMetadataValue(byte[] key)

Description copied from interface: StoreFile

Only used by the Striped Compaction Policy

Specified by:

getMetadataValue in interface StoreFile

Returns:

value associated with the metadata key

isBulkLoadResult

public boolean isBulkLoadResult()

Description copied from interface: StoreFile

Check if this storefile was created by bulk load. When a hfile is bulk loaded into HBase, we append '_SeqId_<id-when-loaded>' to the hfile name, unless "hbase.mapreduce.bulkload.assign.sequenceNumbers" is explicitly turned off. If "hbase.mapreduce.bulkload.assign.sequenceNumbers" is turned off, fall back to BULKLOAD_TIME_KEY.

Specified by:

isBulkLoadResult in interface StoreFile

Returns:

true if this storefile was created by bulk load.

isCompactedAway

public boolean isCompactedAway()

Specified by:

isCompactedAway in interface StoreFile

getRefCount

public int getRefCount()

isReferencedInReads

public boolean isReferencedInReads()

Specified by:

isReferencedInReads in interface StoreFile

Returns:

true if the file is still used in reads

getBulkLoadTimestamp

public OptionalLong getBulkLoadTimestamp()

Description copied from interface: StoreFile

Return the timestamp at which this bulk load file was generated.

Specified by:

getBulkLoadTimestamp in interface StoreFile

getHDFSBlockDistribution

public HDFSBlocksDistribution getHDFSBlockDistribution()

Specified by:

getHDFSBlockDistribution in interface StoreFile

Returns:

the cached value of HDFS blocks distribution. The cached value is calculated when store file is opened.

initReader

public void initReader()
                throws IOException

Description copied from interface: StoreFile

Initialize the reader used for pread.

Specified by:

initReader in interface StoreFile

Throws:

IOException

getPreadScanner

public StoreFileScanner getPreadScanner(boolean cacheBlocks,
                                        long readPt,
                                        long scannerOrder,
                                        boolean canOptimizeForNonNullColumn)

Description copied from interface: StoreFile

Must be called after initReader.

Specified by:

getPreadScanner in interface StoreFile

getStreamScanner

public StoreFileScanner getStreamScanner(boolean canUseDropBehind,
                                         boolean cacheBlocks,
                                         boolean isCompaction,
                                         long readPt,
                                         long scannerOrder,
                                         boolean canOptimizeForNonNullColumn)
                                  throws IOException

Specified by:

getStreamScanner in interface StoreFile

Throws:

IOException

getReader

public StoreFileReader getReader()

Specified by:

getReader in interface StoreFile

Returns:

Current reader. Must call initReader first else returns null.

See Also:

StoreFile.initReader()

closeReader

public void closeReader(boolean evictOnClose)
                 throws IOException

Specified by:

closeReader in interface StoreFile

Parameters:

evictOnClose - whether to evict blocks belonging to this file

Throws:

IOException

markCompactedAway

public void markCompactedAway()

Description copied from interface: StoreFile

Marks the status of the file as compactedAway.

Specified by:

markCompactedAway in interface StoreFile

deleteReader

public void deleteReader()
                  throws IOException

Description copied from interface: StoreFile

Delete this file

Specified by:

deleteReader in interface StoreFile

Throws:

IOException

toString

public String toString()

Overrides:

toString in class Object

toStringDetailed

public String toStringDetailed()

Specified by:

toStringDetailed in interface StoreFile

Returns:

a length description of this StoreFile, suitable for debug output

getMinimumTimestamp

public OptionalLong getMinimumTimestamp()

Specified by:

getMinimumTimestamp in interface StoreFile

getMaximumTimestamp

public OptionalLong getMaximumTimestamp()

Specified by:

getMaximumTimestamp in interface StoreFile