de.gsi.dataset.serializer.spi

Class BinarySerialiser

java.lang.Object

de.gsi.dataset.serializer.spi.BinarySerialiser

Direct Known Subclasses:

BinaryArrayFactory

public class BinarySerialiser
extends Object

Generic binary serialiser aimed at efficiently transferring data between server/client and in particular between Java/C++/web-based programs.

There are two default backing buffer implementations (FastByteBuffer and ByteBuffer), but can be extended/replaced with any other buffer is also possible provided it implements the IoBuffer interface.

The default serialisable data types are defined in DataType and include definitions for

• primitives (byte, short, ..., float, double, and String), and
• arrays thereof (ie. byte[], short[], ..., float[], double[], and String[]), as well as
• complex objects implementing Collections (ie. Set, List, Queues), Enums or Maps.

Any other complex data objects can be stored/extended using the OTHER sub-type. N.B. Multi-dimensional arrays are handled through one-dimensional striding arrays with the additional infos on number of dimensions and size for each individual dimension.

raw-byte level protocol: above data items are stored as follows:

 * header info:   [ start marker ] + 
                      [ "#file producer : ": String ] + 
                      [ clear text serialiser name: String ] + // e.g. "de.gsi.dataset.serializer.spi.BinarySerialiser"
                      [ "\n":String ] + 
                      [ 1 byte - major protocol version ] +
                      [ 1 byte - minor protocol version ] +
                      [ 1 byte - micro protocol version ] // micro: non API-changing bug fixes in implementation 
 * start marker:  [ String - field name ][ 1 byte - uniqueType (0x00) ]
 * String:        [ 4 bytes (int) - length (including termination) ][ n bytes based on latin1 encoding ]
 * field header:  [ String - field name ] + [ 1 byte - uniqueType ]
 * primitives:    [ field header  ] + [ 1-8 bytes depending on DataType ]
 * prim. arrays:  [ array header  ] + [   ]=1-8 bytes x N_i or more - array data depending on variable DataType ]
 * boxed arrays:  as above but each element cast to corresponding primitive type
 * array header:  [ field header  ] + 
                      [4 bytes - number of bytes to skip until data end (-1: uninitialised) ] +
                      [4 bytes - number of dimensions N_d ] + 
                      [4 bytes x N_d - vector sizes for each dimension N_i ]  
 * Collection[E]:
 * List[]:
 * Queue[E]:
 * Set[E]:        [ array header (uniqueType= one of the Collection type IDs) ] + 
                      [ 1 byte - uniqueType of E ] + [  n bytes - array of E cast to primitive type and/or string ]
 * Map[K,V]:      [ array header (uniqueType=0xCB) ] + [ 1 byte - uniqueType of K ] +  [ 1 byte - uniqueType of V ] +
                      [ n bytes - array of K cast to primitive type and/or string ] + 
                      [ n bytes - array of V cast to primitive type and/or string ]
 * OTHER          [ field header - uniqueByte = 0xFD ] + 
                      [ 4 bytes - number of bytes to skip until data end (-1: uninitialised) ] +
                      [ 1 byte - uniqueType -- custom definition ]
                      [ n bytes - custom serialisation definition ]
 * end marker:    [ String - field name ][ 1 byte - uniqueType (0xFE) ]
 
 * nesting or sub-structures (ie. POJOs with sub-classes) can be achieved via:
 [  start marker - field name == nesting context1 ] 
   [  start marker - field name == nesting context2 ]
    ... 
   [  end marker - field name == nesting context2 (optional name) ]
 [  end marker - field name == nesting context1 (optional name) ]
 
 with
 T: being a generic list parameter outlined in DataType
 K: being a generic key parameter outlined in DataType
 V: being a generic value parameter outlined in DataType
 

Author:

rstein

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
class	BinarySerialiser.HeaderInfo

Field Summary

Fields

Modifier and Type	Field and Description
protected static BinarySerialiser	SELF
protected static BinarySerialiser.HeaderInfo	THIS_HEADER
static byte	VERSION_MAJOR
static byte	VERSION_MICRO
static byte	VERSION_MINOR

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	BinarySerialiser()

Method Summary

Modifier and Type	Method and Description
static void	adjustDataByteSizeBlock(IoBuffer buffer, long sizeMarkerStart)
static BinarySerialiser.HeaderInfo	checkHeaderInfo(IoBuffer readBuffer)
static int[]	getArrayDimensions(IoBuffer readBuffer)
static boolean	getBoolean(IoBuffer readBuffer)
static boolean[]	getBooleanArray(IoBuffer readBuffer)
static int	getBufferIncrements()
static byte	getByte(IoBuffer readBuffer)
static byte[]	getByteArray(IoBuffer readBuffer)
static char	getCharacter(IoBuffer readBuffer)
static char[]	getCharArray(IoBuffer readBuffer)
static <E> Collection<E>	getCollection(IoBuffer readBuffer, Collection<E> collection)
static double	getDouble(IoBuffer readBuffer)
static double[]	getDoubleArray(IoBuffer readBuffer)
static double[]	getDoubleArray(IoBuffer readBuffer, DataType dataType)
static <E extends Enum<E>> Enum<E>	getEnum(IoBuffer readBuffer, Enum<E> enumeration)
static String	getEnumTypeList(IoBuffer readBuffer)
static FieldHeader	getFieldHeader(IoBuffer readBuffer)
static float	getFloat(IoBuffer readBuffer)
static float[]	getFloatArray(IoBuffer readBuffer)
protected static Object[]	getGenericArrayAsBoxedPrimitive(IoBuffer readBuffer, DataType dataType)
static int[]	getIntArray(IoBuffer readBuffer)
static int	getInteger(IoBuffer readBuffer)
static <E> List<E>	getList(IoBuffer readBuffer, List<E> collection)
static long	getLong(IoBuffer readBuffer)
static long[]	getLongArray(IoBuffer readBuffer)
static <K,V> Map<K,V>	getMap(IoBuffer readBuffer, Map<K,V> map)
protected static int	getNumberOfElements(int[] dimensions)
static <E> Queue<E>	getQueue(IoBuffer readBuffer, Queue<E> collection)
static <E> Set<E>	getSet(IoBuffer readBuffer, Set<E> collection)
static short	getShort(IoBuffer readBuffer)
static short[]	getShortArray(IoBuffer readBuffer)
static String	getString(IoBuffer readBuffer)
static String[]	getStringArray(IoBuffer readBuffer)
static FieldHeader	parseIoStream(IoBuffer buffer)
protected static void	parseIoStream(IoBuffer buffer, FieldHeader fieldRoot, int recursionDepth)
static void	put(IoBuffer buffer, String fieldName, boolean value)
static void	put(IoBuffer buffer, String fieldName, boolean[] arrayValue)
static void	put(IoBuffer buffer, String fieldName, boolean[] arrayValue, int[] dims)
static void	put(IoBuffer buffer, String fieldName, byte value)
static void	put(IoBuffer buffer, String fieldName, byte[] arrayValue)
static void	put(IoBuffer buffer, String fieldName, byte[] arrayValue, int[] dims)
static void	put(IoBuffer buffer, String fieldName, char value)
static void	put(IoBuffer buffer, String fieldName, char[] arrayValue)
static void	put(IoBuffer buffer, String fieldName, char[] arrayValue, int[] dims)
static <E> void	put(IoBuffer buffer, String fieldName, Collection<E> collection)
static void	put(IoBuffer buffer, String fieldName, double value)
static void	put(IoBuffer buffer, String fieldName, double[] arrayValue)
static void	put(IoBuffer buffer, String fieldName, double[] arrayValue, int[] dims)
static void	put(IoBuffer buffer, String fieldName, Enum<?> enumeration)
static void	put(IoBuffer buffer, String fieldName, float value)
static void	put(IoBuffer buffer, String fieldName, float[] arrayValue)
static void	put(IoBuffer buffer, String fieldName, float[] arrayValue, int[] dims)
static void	put(IoBuffer buffer, String fieldName, int value)
static void	put(IoBuffer buffer, String fieldName, int[] arrayValue)
static void	put(IoBuffer buffer, String fieldName, int[] arrayValue, int[] dims)
static void	put(IoBuffer buffer, String fieldName, long value)
static void	put(IoBuffer buffer, String fieldName, long[] arrayValue)
static void	put(IoBuffer buffer, String fieldName, long[] arrayValue, int[] dims)
static <K,V> void	put(IoBuffer buffer, String fieldName, Map<K,V> map)
static void	put(IoBuffer buffer, String fieldName, short value)
static void	put(IoBuffer buffer, String fieldName, short[] arrayValue)
static void	put(IoBuffer buffer, String fieldName, short[] arrayValue, int[] dims)
static void	put(IoBuffer buffer, String fieldName, String value)
static void	put(IoBuffer buffer, String fieldName, String[] arrayValue)
static void	put(IoBuffer buffer, String fieldName, String[] arrayValue, int[] dims)
static long	putArrayHeader(IoBuffer buffer, String fieldName, DataType dataType, int[] dims, int nElements)
static void	putEndMarker(IoBuffer buffer, String markerName)
static void	putFieldHeader(IoBuffer buffer, String fieldName, DataType dataType)
static void	putFieldHeader(IoBuffer buffer, String fieldName, DataType dataType, int additionalSize)
static void	putGenericArrayAsPrimitive(IoBuffer buffer, DataType dataType, Object[] data, int nToCopy)
static void	putHeaderInfo(IoBuffer buffer) Adds header and version information
static void	putStartMarker(IoBuffer buffer, String markerName)
static void	setBufferIncrements(int bufferIncrements)
protected static void	swallowRest(IoBuffer readBuffer, FieldHeader fieldHeader)
protected static double[]	toDoubles(boolean[] input) Deprecated.
protected static double[]	toDoubles(byte[] input) Deprecated.
protected static double[]	toDoubles(char[] input) Deprecated.
protected static double[]	toDoubles(float[] input) Deprecated.
protected static double[]	toDoubles(int[] input) Deprecated.
protected static double[]	toDoubles(long[] input) Deprecated.
protected static double[]	toDoubles(short[] input) Deprecated.
protected static double[]	toDoubles(String[] input) Deprecated.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

VERSION_MAJOR

public static final byte VERSION_MAJOR

See Also:

Constant Field Values

VERSION_MINOR

public static final byte VERSION_MINOR

See Also:

Constant Field Values

VERSION_MICRO

public static final byte VERSION_MICRO

See Also:

Constant Field Values

SELF

protected static final BinarySerialiser SELF

THIS_HEADER

protected static final BinarySerialiser.HeaderInfo THIS_HEADER

Constructor Detail

BinarySerialiser

protected BinarySerialiser()

Method Detail

adjustDataByteSizeBlock

public static void adjustDataByteSizeBlock(IoBuffer buffer,
                                           long sizeMarkerStart)

checkHeaderInfo

public static BinarySerialiser.HeaderInfo checkHeaderInfo(IoBuffer readBuffer)

getArrayDimensions

public static int[] getArrayDimensions(IoBuffer readBuffer)

getBoolean

public static boolean getBoolean(IoBuffer readBuffer)

getBooleanArray

public static boolean[] getBooleanArray(IoBuffer readBuffer)

getBufferIncrements

public static int getBufferIncrements()

getByte

public static byte getByte(IoBuffer readBuffer)

getByteArray

public static byte[] getByteArray(IoBuffer readBuffer)

getCharacter

public static char getCharacter(IoBuffer readBuffer)

getCharArray

public static char[] getCharArray(IoBuffer readBuffer)

getCollection

public static <E> Collection<E> getCollection(IoBuffer readBuffer,
                                              Collection<E> collection)

getDouble

public static double getDouble(IoBuffer readBuffer)

getDoubleArray

public static double[] getDoubleArray(IoBuffer readBuffer)

getDoubleArray

public static double[] getDoubleArray(IoBuffer readBuffer,
                                      DataType dataType)

getEnum

public static <E extends Enum<E>> Enum<E> getEnum(IoBuffer readBuffer,
                                                  Enum<E> enumeration)

getEnumTypeList

public static String getEnumTypeList(IoBuffer readBuffer)

getFieldHeader

public static FieldHeader getFieldHeader(IoBuffer readBuffer)

getFloat

public static float getFloat(IoBuffer readBuffer)

getFloatArray

public static float[] getFloatArray(IoBuffer readBuffer)

getGenericArrayAsBoxedPrimitive

protected static Object[] getGenericArrayAsBoxedPrimitive(IoBuffer readBuffer,
                                                          DataType dataType)

getIntArray

public static int[] getIntArray(IoBuffer readBuffer)

getInteger

public static int getInteger(IoBuffer readBuffer)

getList

public static <E> List<E> getList(IoBuffer readBuffer,
                                  List<E> collection)

getLong

public static long getLong(IoBuffer readBuffer)

getLongArray

public static long[] getLongArray(IoBuffer readBuffer)

getMap

public static <K,V> Map<K,V> getMap(IoBuffer readBuffer,
                                    Map<K,V> map)

getNumberOfElements

protected static int getNumberOfElements(int[] dimensions)

getQueue

public static <E> Queue<E> getQueue(IoBuffer readBuffer,
                                    Queue<E> collection)

getSet

public static <E> Set<E> getSet(IoBuffer readBuffer,
                                Set<E> collection)

getShort

public static short getShort(IoBuffer readBuffer)

getShortArray

public static short[] getShortArray(IoBuffer readBuffer)

getString

public static String getString(IoBuffer readBuffer)

getStringArray

public static String[] getStringArray(IoBuffer readBuffer)

parseIoStream

public static FieldHeader parseIoStream(IoBuffer buffer)

parseIoStream

protected static void parseIoStream(IoBuffer buffer,
                                    FieldHeader fieldRoot,
                                    int recursionDepth)

put

public static void put(IoBuffer buffer,
                       String fieldName,
                       boolean value)

put

public static void put(IoBuffer buffer,
                       String fieldName,
                       boolean[] arrayValue)

put

public static void put(IoBuffer buffer,
                       String fieldName,
                       boolean[] arrayValue,
                       int[] dims)

put

public static void put(IoBuffer buffer,
                       String fieldName,
                       byte value)

put

public static void put(IoBuffer buffer,
                       String fieldName,
                       byte[] arrayValue)

put

public static void put(IoBuffer buffer,
                       String fieldName,
                       byte[] arrayValue,
                       int[] dims)

put

public static void put(IoBuffer buffer,
                       String fieldName,
                       char value)

put

public static void put(IoBuffer buffer,
                       String fieldName,
                       char[] arrayValue)

put

public static void put(IoBuffer buffer,
                       String fieldName,
                       char[] arrayValue,
                       int[] dims)

put

public static <E> void put(IoBuffer buffer,
                           String fieldName,
                           Collection<E> collection)

put

public static void put(IoBuffer buffer,
                       String fieldName,
                       double value)

put

public static void put(IoBuffer buffer,
                       String fieldName,
                       double[] arrayValue)

put

public static void put(IoBuffer buffer,
                       String fieldName,
                       double[] arrayValue,
                       int[] dims)

put

public static void put(IoBuffer buffer,
                       String fieldName,
                       Enum<?> enumeration)

put

public static void put(IoBuffer buffer,
                       String fieldName,
                       float value)

put

public static void put(IoBuffer buffer,
                       String fieldName,
                       float[] arrayValue)

put

public static void put(IoBuffer buffer,
                       String fieldName,
                       float[] arrayValue,
                       int[] dims)

put

public static void put(IoBuffer buffer,
                       String fieldName,
                       int value)

put

public static void put(IoBuffer buffer,
                       String fieldName,
                       int[] arrayValue)

put

public static void put(IoBuffer buffer,
                       String fieldName,
                       int[] arrayValue,
                       int[] dims)

put

public static void put(IoBuffer buffer,
                       String fieldName,
                       long value)

put

public static void put(IoBuffer buffer,
                       String fieldName,
                       long[] arrayValue)

put

public static void put(IoBuffer buffer,
                       String fieldName,
                       long[] arrayValue,
                       int[] dims)

put

public static <K,V> void put(IoBuffer buffer,
                             String fieldName,
                             Map<K,V> map)

put

public static void put(IoBuffer buffer,
                       String fieldName,
                       short value)

put

public static void put(IoBuffer buffer,
                       String fieldName,
                       short[] arrayValue)

put

public static void put(IoBuffer buffer,
                       String fieldName,
                       short[] arrayValue,
                       int[] dims)

put

public static void put(IoBuffer buffer,
                       String fieldName,
                       String value)

put

public static void put(IoBuffer buffer,
                       String fieldName,
                       String[] arrayValue)

put

public static void put(IoBuffer buffer,
                       String fieldName,
                       String[] arrayValue,
                       int[] dims)

putArrayHeader

public static long putArrayHeader(IoBuffer buffer,
                                  String fieldName,
                                  DataType dataType,
                                  int[] dims,
                                  int nElements)

putEndMarker

public static void putEndMarker(IoBuffer buffer,
                                String markerName)

putFieldHeader

public static void putFieldHeader(IoBuffer buffer,
                                  String fieldName,
                                  DataType dataType)

putFieldHeader

public static void putFieldHeader(IoBuffer buffer,
                                  String fieldName,
                                  DataType dataType,
                                  int additionalSize)

putGenericArrayAsPrimitive

public static void putGenericArrayAsPrimitive(IoBuffer buffer,
                                              DataType dataType,
                                              Object[] data,
                                              int nToCopy)

putHeaderInfo

public static void putHeaderInfo(IoBuffer buffer)

Adds header and version information

Parameters:

buffer - to use for serialisation

putStartMarker

public static void putStartMarker(IoBuffer buffer,
                                  String markerName)

setBufferIncrements

public static void setBufferIncrements(int bufferIncrements)

swallowRest

protected static void swallowRest(IoBuffer readBuffer,
                                  FieldHeader fieldHeader)

toDoubles

@Deprecated
protected static double[] toDoubles(boolean[] input)

Deprecated.

toDoubles

@Deprecated
protected static double[] toDoubles(byte[] input)

Deprecated.

toDoubles

@Deprecated
protected static double[] toDoubles(char[] input)

Deprecated.

toDoubles

@Deprecated
protected static double[] toDoubles(float[] input)

Deprecated.

toDoubles

@Deprecated
protected static double[] toDoubles(int[] input)

Deprecated.

toDoubles

@Deprecated
protected static double[] toDoubles(long[] input)

Deprecated.

toDoubles

@Deprecated
protected static double[] toDoubles(short[] input)

Deprecated.

toDoubles

@Deprecated
protected static double[] toDoubles(String[] input)

Deprecated.