de.gsi.dataset.utils

Class DataSetSerialiser

java.lang.Object

de.gsi.dataset.utils.DataSetUtilsHelper

de.gsi.dataset.utils.DataSetSerialiser

public class DataSetSerialiser
extends DataSetUtilsHelper

Class to efficiently serialise and de-serialise DataSet objects into binary byte arrays. The performance can be tuned through:

• using floats (ie. memory-IO vs network-IO bound serialisation), or
• via setDataLablesSerialised(boolean) (default: false) to control whether data labels and styles shall be processed
• via setMetaDataSerialised(boolean) (default: true) to control whether meta data shall be processed

Author:

rstein

Field Summary

Fields inherited from class de.gsi.dataset.utils.DataSetUtilsHelper

BYTE_ARRAY_CACHE_LOCK, byteArrayCache, STRING_BUFFER_CACHE_LOCK, stringBuilderCache

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	DataSetSerialiser()

Method Summary

Modifier and Type	Method and Description
protected static boolean	checkFieldCompatibility(BinarySerialiser.FieldHeader fieldHeader, String fieldName, DataType requireDataType)
static boolean	isDataLablesSerialised()
static boolean	isMetaDataSerialised()
static DataSet	readDataSetFromByteArray(FastByteBuffer readBuffer) Read a Dataset from a byte array containing comma separated values. The data format is a custom extension of csv with an additional #-commented Metadata Header and a $-commented column header.
static void	setDataLablesSerialised(boolean state)
static void	setMetaDataSerialised(boolean state)
protected static void	swallowRest(FastByteBuffer readBuffer, BinarySerialiser.FieldHeader fieldHeader)
protected static double[]	toDoubles(float[] input)
protected static float[]	toFloats(double[] input)
protected static void	writeDataLabelsToStream(FastByteBuffer buffer, DataSet dataSet)
static void	writeDataSetToByteArray(DataSet dataSet, FastByteBuffer buffer, boolean asFloat) Write data set into byte buffer.
protected static void	writeHeaderDataToStream(FastByteBuffer buffer, DataSet dataSet)
protected static void	writeMetaDataToStream(FastByteBuffer buffer, DataSet dataSet)
protected static void	writeNumericBinaryDataToBuffer(FastByteBuffer buffer, DataSet dataSet, boolean asFloat)

Methods inherited from class de.gsi.dataset.utils.DataSetUtilsHelper

getCachedDoubleArray, getCachedStringBuilder, integralSimple, mean, readDoubleArrayFromBuffer, release, release, rootMeanSquare, toDoubleArray, toFloatArray, writeDoubleArrayAsFloatToByteBuffer, writeDoubleArrayToByteBuffer

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DataSetSerialiser

protected DataSetSerialiser()

Method Detail

isMetaDataSerialised

public static boolean isMetaDataSerialised()

setMetaDataSerialised

public static void setMetaDataSerialised(boolean state)

isDataLablesSerialised

public static boolean isDataLablesSerialised()

setDataLablesSerialised

public static void setDataLablesSerialised(boolean state)

writeDataSetToByteArray

public static void writeDataSetToByteArray(DataSet dataSet,
                                           FastByteBuffer buffer,
                                           boolean asFloat)

Write data set into byte buffer.

Parameters:

dataSet - The DataSet to export

buffer - byte output buffer (N.B. keep caching this object)

asFloat - true: encode data as binary floats (smaller size, performance), or false as double (better precision)

writeHeaderDataToStream

protected static void writeHeaderDataToStream(FastByteBuffer buffer,
                                              DataSet dataSet)

writeMetaDataToStream

protected static void writeMetaDataToStream(FastByteBuffer buffer,
                                            DataSet dataSet)

writeDataLabelsToStream

protected static void writeDataLabelsToStream(FastByteBuffer buffer,
                                              DataSet dataSet)

writeNumericBinaryDataToBuffer

protected static void writeNumericBinaryDataToBuffer(FastByteBuffer buffer,
                                                     DataSet dataSet,
                                                     boolean asFloat)

Parameters:

buffer - FastByteBuffer to write binary data into

dataSet - to be exported

asFloat - true use 32-bit floats (less memory, faster transfer) instead of 64-bit doubles (DataSet default, higher precision)

checkFieldCompatibility

protected static boolean checkFieldCompatibility(BinarySerialiser.FieldHeader fieldHeader,
                                                 String fieldName,
                                                 DataType requireDataType)

readDataSetFromByteArray

public static DataSet readDataSetFromByteArray(FastByteBuffer readBuffer)

Read a Dataset from a byte array containing comma separated values.
The data format is a custom extension of csv with an additional #-commented Metadata Header and a $-commented column header. Expects the following columns in this order to be present: index, x, y, eyn, eyp.

Parameters:

readBuffer - FastByteBuffer (encapsulates byte array).

Returns:

DataSet with the data and metadata read from the file

swallowRest

protected static void swallowRest(FastByteBuffer readBuffer,
                                  BinarySerialiser.FieldHeader fieldHeader)

toFloats

protected static float[] toFloats(double[] input)

toDoubles

protected static double[] toDoubles(float[] input)