org.nustaq.offheap

Class FSTBinaryOffheapMap

java.lang.Object

org.nustaq.offheap.FSTBinaryOffheapMap

Direct Known Subclasses:

FSTCodedOffheapMap

public class FSTBinaryOffheapMap
extends Object

Baseclass of offheap maps. FST OffHeap Maps enable to store key/value pairs in offheap memory. Additionally it provides and iterator interface for all values. In order to also iterate keys, add each key to its value object, as this OffHeap map does not support iteration of keys out of the box. See subclasses for directly applicable classes (E.g. FSTStringOffheapMap) The base is a generic bytesource to bytesource map. Note that key should be as short as possible (4-20 bytes) and should have their most modified digit at the last character of their value. e.g. [0,0,0,0,123,44] where '44' changes with each new key. Else on-heap memory consumption will grow. Performance of lookup degrades with growing key size. In case entries are updated frequently with values of different size, avoid fragementation by adding extra space to each entry. Override getEntryLengthForContentLength for this.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static interface	FSTBinaryOffheapMap.KeyValIter

Field Summary

Fields

Modifier and Type	Field and Description
protected MallocBytezAllocator	alloc
protected long	bytezOffset
static int	CORE_HEADER_LEN
static int	CUSTOM_FILEHEADER_LEN
protected Bytez	customHeader
static int	FILE_HEADER_LEN
protected FreeList	freeList
static long	GB
protected OffHeapByteTree	index
static int	KEY_OFFSET_IN_HEADER
protected int	keyLen
static long	MB
protected Bytez	memory
protected int	numElem

Constructor Summary

Constructors

Constructor and Description
FSTBinaryOffheapMap(int keyLen, long sizeMemBytes, int numberOfElems)
FSTBinaryOffheapMap(String mappedFile, int keyLen, long sizeMemBytes, int numberOfElems)

Method Summary

Methods

Modifier and Type	Method and Description
protected long	addEntry(ByteSource key, ByteSource value)
protected void	addToFreeList(long offset)
FSTBinaryOffheapMap.KeyValIter	binaryKeys()
Iterator<ByteSource>	binaryValues()
protected void	decElems()
void	dumpIndexStats()
protected void	finalize()
void	free()
BytezByteSource	getBinary(ByteSource key) get an entry.
int	getCapacityMB()
protected int	getContentLenFromHeader(long off)
Bytez	getCustomFileHeader()
protected int	getEntryLengthForContentLength(long lengthOfEntry) called upon add, allows to reserve extra space for later growth per entry
long	getFreeMem()
protected int	getHeaderLen()
protected int	getLenFromHeader(long off)
int	getSize()
long	getUsedMem()
protected void	incElems()
protected void	init(int keyLen, long sizeMemBytes, int numberOfElems)
protected void	initFromFile(String file, int keyLen, long sizeMemBytes, int numberOfElems)
String	printBinaryKey(ByteSource key)
void	putBinary(ByteSource key, ByteSource value)
void	removeBinary(ByteSource key) remove the key from the binary map
protected void	removeEntry(long offset)
protected void	setEntry(long off, int entryLen, ByteSource value)
protected void	writeEntryHeader(long offset, int entryLen, int contentLen, boolean removed)

Methods inherited from class java.lang.Object

clone, equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

MB

public static final long MB

See Also:

Constant Field Values

GB

public static final long GB

See Also:

Constant Field Values

CUSTOM_FILEHEADER_LEN

public static final int CUSTOM_FILEHEADER_LEN

See Also:

Constant Field Values

CORE_HEADER_LEN

public static final int CORE_HEADER_LEN

See Also:

Constant Field Values

FILE_HEADER_LEN

public static final int FILE_HEADER_LEN

See Also:

Constant Field Values

KEY_OFFSET_IN_HEADER

public static final int KEY_OFFSET_IN_HEADER

See Also:

Constant Field Values

index

protected OffHeapByteTree index

memory

protected Bytez memory

customHeader

protected Bytez customHeader

alloc

protected MallocBytezAllocator alloc

numElem

protected int numElem

keyLen

protected int keyLen

bytezOffset

protected long bytezOffset

freeList

protected FreeList freeList

Constructor Detail

FSTBinaryOffheapMap

public FSTBinaryOffheapMap(String mappedFile,
                   int keyLen,
                   long sizeMemBytes,
                   int numberOfElems)
                    throws Exception

Throws:

Exception

FSTBinaryOffheapMap

public FSTBinaryOffheapMap(int keyLen,
                   long sizeMemBytes,
                   int numberOfElems)

Method Detail

getCustomFileHeader

public Bytez getCustomFileHeader()

initFromFile

protected void initFromFile(String file,
                int keyLen,
                long sizeMemBytes,
                int numberOfElems)
                     throws Exception

Throws:

Exception

init

protected void init(int keyLen,
        long sizeMemBytes,
        int numberOfElems)

finalize

protected void finalize()
                 throws Throwable

Overrides:

finalize in class Object

Throws:

Throwable

free

public void free()

putBinary

public void putBinary(ByteSource key,
             ByteSource value)

removeEntry

protected void removeEntry(long offset)

addToFreeList

protected void addToFreeList(long offset)

setEntry

protected void setEntry(long off,
            int entryLen,
            ByteSource value)

addEntry

protected long addEntry(ByteSource key,
            ByteSource value)

getBinary

public BytezByteSource getBinary(ByteSource key)

get an entry. the returned ByteSource must be processed immediately as it will be reused internally on next get Warning: Concurrent Modification (e.g. add remove elements during iteration) is NOT SUPPORTED and NOT CHECKED. Collect keys to change inside iteration and perform changes after iteration is finished.

Parameters:

key -

Returns:

removeBinary

public void removeBinary(ByteSource key)

remove the key from the binary map

Parameters:

key -

decElems

protected void decElems()

incElems

protected void incElems()

getEntryLengthForContentLength

protected int getEntryLengthForContentLength(long lengthOfEntry)

called upon add, allows to reserve extra space for later growth per entry

Parameters:

lengthOfEntry -

Returns:

writeEntryHeader

protected void writeEntryHeader(long offset,
                    int entryLen,
                    int contentLen,
                    boolean removed)

getHeaderLen

protected int getHeaderLen()

getLenFromHeader

protected int getLenFromHeader(long off)

getContentLenFromHeader

protected int getContentLenFromHeader(long off)

binaryValues

public Iterator<ByteSource> binaryValues()

printBinaryKey

public String printBinaryKey(ByteSource key)

binaryKeys

public FSTBinaryOffheapMap.KeyValIter binaryKeys()

getFreeMem

public long getFreeMem()

getUsedMem

public long getUsedMem()

getCapacityMB

public int getCapacityMB()

getSize

public int getSize()

dumpIndexStats

public void dumpIndexStats()