Package de.gsi.dataset.serializer.spi

Class BinarySerialiser

  • Direct Known Subclasses:
    BinaryArrayFactory
    public class BinarySerialiser
extends java.lang.Object
    Generic binary serialiser aimed at efficiently transferring data between server/client and in particular between Java/C++/web-based programs.

    There are two default backing buffer implementations (FastByteBuffer and ByteBuffer), but can be extended/replaced with any other buffer is also possible provided it implements the IoBuffer interface.

    The default serialisable data types are defined in DataType and include definitions for

    • primitives (byte, short, ..., float, double, and String), and
    • arrays thereof (ie. byte[], short[], ..., float[], double[], and String[]), as well as
    • complex objects implementing Collections (ie. Set, List, Queues), Enums or Maps.
    Any other complex data objects can be stored/extended using the OTHER sub-type. N.B. Multi-dimensional arrays are handled through one-dimensional striding arrays with the additional infos on number of dimensions and size for each individual dimension.

    raw-byte level protocol: above data items are stored as follows: 

    
 * header info:   [ start marker ] + 
                      [ "#file producer : ": String ] + 
                      [ clear text serialiser name: String ] + // e.g. "de.gsi.dataset.serializer.spi.BinarySerialiser"
                      [ "\n":String ] + 
                      [ 1 byte - major protocol version ] +
                      [ 1 byte - minor protocol version ] +
                      [ 1 byte - micro protocol version ] // micro: non API-changing bug fixes in implementation 
 * start marker:  [ String - field name ][ 1 byte - uniqueType (0x00) ]
 * String:        [ 4 bytes (int) - length (including termination) ][ n bytes based on latin1 encoding ]
 * field header:  [ String - field name ] + [ 1 byte - uniqueType ]
 * primitives:    [ field header  ] + [ 1-8 bytes depending on DataType ]
 * prim. arrays:  [ array header  ] + [   ]=1-8 bytes x N_i or more - array data depending on variable DataType ]
 * boxed arrays:  as above but each element cast to corresponding primitive type
 * array header:  [ field header  ] + 
                      [4 bytes - number of bytes to skip until data end (-1: uninitialised) ] +
                      [4 bytes - number of dimensions N_d ] + 
                      [4 bytes x N_d - vector sizes for each dimension N_i ]  
 * Collection[E]:
 * List[]:
 * Queue[E]:
 * Set[E]:        [ array header (uniqueType= one of the Collection type IDs) ] + 
                      [ 1 byte - uniqueType of E ] + [  n bytes - array of E cast to primitive type and/or string ]
 * Map[K,V]:      [ array header (uniqueType=0xCB) ] + [ 1 byte - uniqueType of K ] +  [ 1 byte - uniqueType of V ] +
                      [ n bytes - array of K cast to primitive type and/or string ] + 
                      [ n bytes - array of V cast to primitive type and/or string ]
 * OTHER          [ field header - uniqueByte = 0xFD ] + 
                      [ 4 bytes - number of bytes to skip until data end (-1: uninitialised) ] +
                      [ 1 byte - uniqueType -- custom definition ]
                      [ n bytes - custom serialisation definition ]
 * end marker:    [ String - field name ][ 1 byte - uniqueType (0xFE) ]
 
 * nesting or sub-structures (ie. POJOs with sub-classes) can be achieved via:
 [  start marker - field name == nesting context1 ] 
   [  start marker - field name == nesting context2 ]
    ... 
   [  end marker - field name == nesting context2 (optional name) ]
 [  end marker - field name == nesting context1 (optional name) ]
 
 with
 T: being a generic list parameter outlined in DataType
 K: being a generic key parameter outlined in DataType
 V: being a generic value parameter outlined in DataType
    Author:
    rstein

    • Constructor Detail

      • BinarySerialiser

        protected BinarySerialiser()

    • Method Detail

      • adjustDataByteSizeBlock

        public static void adjustDataByteSizeBlock​(IoBuffer buffer,
                                           long sizeMarkerStart)

      • getArrayDimensions

        public static int[] getArrayDimensions​(IoBuffer readBuffer)

      • getBoolean

        public static boolean getBoolean​(IoBuffer readBuffer)

      • getBooleanArray

        public static boolean[] getBooleanArray​(IoBuffer readBuffer)

      • getBufferIncrements

        public static int getBufferIncrements()

      • getByte

        public static byte getByte​(IoBuffer readBuffer)

      • getByteArray

        public static byte[] getByteArray​(IoBuffer readBuffer)

      • getCharacter

        public static char getCharacter​(IoBuffer readBuffer)

      • getCharArray

        public static char[] getCharArray​(IoBuffer readBuffer)

      • getCollection

        public static <E> java.util.Collection<E> getCollection​(IoBuffer readBuffer,
                                                        java.util.Collection<E> collection)

      • getDouble

        public static double getDouble​(IoBuffer readBuffer)

      • getDoubleArray

        public static double[] getDoubleArray​(IoBuffer readBuffer)

      • getDoubleArray

        public static double[] getDoubleArray​(IoBuffer readBuffer,
                                      DataType dataType)

      • getEnum

        public static <E extends java.lang.Enum<E>> java.lang.Enum<E> getEnum​(IoBuffer readBuffer,
                                                                      java.lang.Enum<E> enumeration)

      • getEnumTypeList

        public static java.lang.String getEnumTypeList​(IoBuffer readBuffer)

      • getFloat

        public static float getFloat​(IoBuffer readBuffer)

      • getFloatArray

        public static float[] getFloatArray​(IoBuffer readBuffer)

      • getGenericArrayAsBoxedPrimitive

        protected static java.lang.Object[] getGenericArrayAsBoxedPrimitive​(IoBuffer readBuffer,
                                                                    DataType dataType)

      • getIntArray

        public static int[] getIntArray​(IoBuffer readBuffer)

      • getInteger

        public static int getInteger​(IoBuffer readBuffer)

      • getList

        public static <E> java.util.List<E> getList​(IoBuffer readBuffer,
                                            java.util.List<E> collection)

      • getLong

        public static long getLong​(IoBuffer readBuffer)

      • getLongArray

        public static long[] getLongArray​(IoBuffer readBuffer)

      • getMap

        public static <K,​V> java.util.Map<K,​V> getMap​(IoBuffer readBuffer,
                                                          java.util.Map<K,​V> map)

      • getNumberOfElements

        protected static int getNumberOfElements​(int[] dimensions)

      • getQueue

        public static <E> java.util.Queue<E> getQueue​(IoBuffer readBuffer,
                                              java.util.Queue<E> collection)

      • getSet

        public static <E> java.util.Set<E> getSet​(IoBuffer readBuffer,
                                          java.util.Set<E> collection)

      • getShort

        public static short getShort​(IoBuffer readBuffer)

      • getShortArray

        public static short[] getShortArray​(IoBuffer readBuffer)

      • getString

        public static java.lang.String getString​(IoBuffer readBuffer)

      • getStringArray

        public static java.lang.String[] getStringArray​(IoBuffer readBuffer)

      • parseIoStream

        protected static void parseIoStream​(IoBuffer buffer,
                                    FieldHeader fieldRoot,
                                    int recursionDepth)

      • put

        public static void put​(IoBuffer buffer,
                       java.lang.String fieldName,
                       boolean value)

      • put

        public static void put​(IoBuffer buffer,
                       java.lang.String fieldName,
                       boolean[] arrayValue)

      • put

        public static void put​(IoBuffer buffer,
                       java.lang.String fieldName,
                       boolean[] arrayValue,
                       int[] dims)

      • put

        public static void put​(IoBuffer buffer,
                       java.lang.String fieldName,
                       byte value)

      • put

        public static void put​(IoBuffer buffer,
                       java.lang.String fieldName,
                       byte[] arrayValue)

      • put

        public static void put​(IoBuffer buffer,
                       java.lang.String fieldName,
                       byte[] arrayValue,
                       int[] dims)

      • put

        public static void put​(IoBuffer buffer,
                       java.lang.String fieldName,
                       char value)

      • put

        public static void put​(IoBuffer buffer,
                       java.lang.String fieldName,
                       char[] arrayValue)

      • put

        public static void put​(IoBuffer buffer,
                       java.lang.String fieldName,
                       char[] arrayValue,
                       int[] dims)

      • put

        public static <E> void put​(IoBuffer buffer,
                           java.lang.String fieldName,
                           java.util.Collection<E> collection)

      • put

        public static void put​(IoBuffer buffer,
                       java.lang.String fieldName,
                       double value)

      • put

        public static void put​(IoBuffer buffer,
                       java.lang.String fieldName,
                       double[] arrayValue)

      • put

        public static void put​(IoBuffer buffer,
                       java.lang.String fieldName,
                       double[] arrayValue,
                       int[] dims)

      • put

        public static void put​(IoBuffer buffer,
                       java.lang.String fieldName,
                       java.lang.Enum<?> enumeration)

      • put

        public static void put​(IoBuffer buffer,
                       java.lang.String fieldName,
                       float value)

      • put

        public static void put​(IoBuffer buffer,
                       java.lang.String fieldName,
                       float[] arrayValue)

      • put

        public static void put​(IoBuffer buffer,
                       java.lang.String fieldName,
                       float[] arrayValue,
                       int[] dims)

      • put

        public static void put​(IoBuffer buffer,
                       java.lang.String fieldName,
                       int value)

      • put

        public static void put​(IoBuffer buffer,
                       java.lang.String fieldName,
                       int[] arrayValue)

      • put

        public static void put​(IoBuffer buffer,
                       java.lang.String fieldName,
                       int[] arrayValue,
                       int[] dims)

      • put

        public static void put​(IoBuffer buffer,
                       java.lang.String fieldName,
                       long value)

      • put

        public static void put​(IoBuffer buffer,
                       java.lang.String fieldName,
                       long[] arrayValue)

      • put

        public static void put​(IoBuffer buffer,
                       java.lang.String fieldName,
                       long[] arrayValue,
                       int[] dims)

      • put

        public static <K,​V> void put​(IoBuffer buffer,
                                   java.lang.String fieldName,
                                   java.util.Map<K,​V> map)

      • put

        public static void put​(IoBuffer buffer,
                       java.lang.String fieldName,
                       short value)

      • put

        public static void put​(IoBuffer buffer,
                       java.lang.String fieldName,
                       short[] arrayValue)

      • put

        public static void put​(IoBuffer buffer,
                       java.lang.String fieldName,
                       short[] arrayValue,
                       int[] dims)

      • put

        public static void put​(IoBuffer buffer,
                       java.lang.String fieldName,
                       java.lang.String value)

      • put

        public static void put​(IoBuffer buffer,
                       java.lang.String fieldName,
                       java.lang.String[] arrayValue)

      • put

        public static void put​(IoBuffer buffer,
                       java.lang.String fieldName,
                       java.lang.String[] arrayValue,
                       int[] dims)

      • putArrayHeader

        public static long putArrayHeader​(IoBuffer buffer,
                                  java.lang.String fieldName,
                                  DataType dataType,
                                  int[] dims,
                                  int nElements)

      • putEndMarker

        public static void putEndMarker​(IoBuffer buffer,
                                java.lang.String markerName)

      • putFieldHeader

        public static void putFieldHeader​(IoBuffer buffer,
                                  java.lang.String fieldName,
                                  DataType dataType)

      • putFieldHeader

        public static void putFieldHeader​(IoBuffer buffer,
                                  java.lang.String fieldName,
                                  DataType dataType,
                                  int additionalSize)

      • putGenericArrayAsPrimitive

        public static void putGenericArrayAsPrimitive​(IoBuffer buffer,
                                              DataType dataType,
                                              java.lang.Object[] data,
                                              int nToCopy)

      • putHeaderInfo

        public static void putHeaderInfo​(IoBuffer buffer)
        Adds header and version information
        Parameters:
        buffer - to use for serialisation

      • putStartMarker

        public static void putStartMarker​(IoBuffer buffer,
                                  java.lang.String markerName)

      • setBufferIncrements

        public static void setBufferIncrements​(int bufferIncrements)

      • swallowRest

        protected static void swallowRest​(IoBuffer readBuffer,
                                  FieldHeader fieldHeader)

      • toDoubles

        @Deprecated
protected static double[] toDoubles​(boolean[] input)
        Deprecated.

      • toDoubles

        @Deprecated
protected static double[] toDoubles​(byte[] input)
        Deprecated.

      • toDoubles

        @Deprecated
protected static double[] toDoubles​(char[] input)
        Deprecated.

      • toDoubles

        @Deprecated
protected static double[] toDoubles​(float[] input)
        Deprecated.

      • toDoubles

        @Deprecated
protected static double[] toDoubles​(int[] input)
        Deprecated.

      • toDoubles

        @Deprecated
protected static double[] toDoubles​(long[] input)
        Deprecated.

      • toDoubles

        @Deprecated
protected static double[] toDoubles​(short[] input)
        Deprecated.

      • toDoubles

        @Deprecated
protected static double[] toDoubles​(java.lang.String[] input)
        Deprecated.