org.apache.flink.contrib.streaming.state

Enum PredefinedOptions

java.lang.Object

java.lang.Enum<PredefinedOptions>

org.apache.flink.contrib.streaming.state.PredefinedOptions

All Implemented Interfaces:

Serializable, Comparable<PredefinedOptions>

public enum PredefinedOptions
extends Enum<PredefinedOptions>

The PredefinedOptions are configuration settings for the RocksDBStateBackend. The various pre-defined choices are configurations that have been empirically determined to be beneficial for performance under different settings.

Some of these settings are based on experiments by the Flink community, some follow guides from the RocksDB project.

Enum Constant Summary

Enum Constants

Enum Constant and Description
DEFAULT Default options for all settings, except that writes are not forced to the disk.
FLASH_SSD_OPTIMIZED Pre-defined options for Flash SSDs.
SPINNING_DISK_OPTIMIZED Pre-defined options for regular spinning hard disks.
SPINNING_DISK_OPTIMIZED_HIGH_MEM Pre-defined options for better performance on regular spinning hard disks, at the cost of a higher memory consumption.

Method Summary

Modifier and Type	Method and Description
abstract org.rocksdb.ColumnFamilyOptions	createColumnOptions() Creates the ColumnFamilyOptionsfor this pre-defined setting.
abstract org.rocksdb.DBOptions	createDBOptions() Creates the DBOptionsfor this pre-defined setting.
static PredefinedOptions	valueOf(String name) Returns the enum constant of this type with the specified name.
static PredefinedOptions[]	values() Returns an array containing the constants of this enum type, in the order they are declared.

Methods inherited from class java.lang.Enum

clone, compareTo, equals, finalize, getDeclaringClass, hashCode, name, ordinal, toString, valueOf

Methods inherited from class java.lang.Object

getClass, notify, notifyAll, wait, wait, wait

Enum Constant Detail

DEFAULT

public static final PredefinedOptions DEFAULT

Default options for all settings, except that writes are not forced to the disk.

Note: Because Flink does not rely on RocksDB data on disk for recovery, there is no need to sync data to stable storage.

SPINNING_DISK_OPTIMIZED

public static final PredefinedOptions SPINNING_DISK_OPTIMIZED

Pre-defined options for regular spinning hard disks.

This constant configures RocksDB with some options that lead empirically to better performance when the machines executing the system use regular spinning hard disks.

The following options are set:

• setCompactionStyle(CompactionStyle.LEVEL)
• setLevelCompactionDynamicLevelBytes(true)
• setIncreaseParallelism(4)
• setUseFsync(false)
• setDisableDataSync(true)
• setMaxOpenFiles(-1)

Note: Because Flink does not rely on RocksDB data on disk for recovery, there is no need to sync data to stable storage.

SPINNING_DISK_OPTIMIZED_HIGH_MEM

public static final PredefinedOptions SPINNING_DISK_OPTIMIZED_HIGH_MEM

Pre-defined options for better performance on regular spinning hard disks, at the cost of a higher memory consumption.

NOTE: These settings will cause RocksDB to consume a lot of memory for block caching and compactions. If you experience out-of-memory problems related to, RocksDB, consider switching back to SPINNING_DISK_OPTIMIZED.

The following options are set:

• setLevelCompactionDynamicLevelBytes(true)
• setTargetFileSizeBase(256 MBytes)
• setMaxBytesForLevelBase(1 GByte)
• setWriteBufferSize(64 MBytes)
• setIncreaseParallelism(4)
• setMinWriteBufferNumberToMerge(3)
• setMaxWriteBufferNumber(4)
• setUseFsync(false)
• setMaxOpenFiles(-1)
• BlockBasedTableConfig.setBlockCacheSize(256 MBytes)
• BlockBasedTableConfigsetBlockSize(128 KBytes)

Note: Because Flink does not rely on RocksDB data on disk for recovery, there is no need to sync data to stable storage.

FLASH_SSD_OPTIMIZED

public static final PredefinedOptions FLASH_SSD_OPTIMIZED

Pre-defined options for Flash SSDs.

This constant configures RocksDB with some options that lead empirically to better performance when the machines executing the system use SSDs.

The following options are set:

• setIncreaseParallelism(4)
• setUseFsync(false)
• setDisableDataSync(true)
• setMaxOpenFiles(-1)

Note: Because Flink does not rely on RocksDB data on disk for recovery, there is no need to sync data to stable storage.

Method Detail

values

public static PredefinedOptions[] values()

Returns an array containing the constants of this enum type, in the order they are declared. This method may be used to iterate over the constants as follows:

for (PredefinedOptions c : PredefinedOptions.values())
    System.out.println(c);

Returns:

an array containing the constants of this enum type, in the order they are declared

valueOf

public static PredefinedOptions valueOf(String name)

Returns the enum constant of this type with the specified name. The string must match exactly an identifier used to declare an enum constant in this type. (Extraneous whitespace characters are not permitted.)

Parameters:

name - the name of the enum constant to be returned.

Returns:

the enum constant with the specified name

Throws:

IllegalArgumentException - if this enum type has no constant with the specified name

NullPointerException - if the argument is null

createDBOptions

public abstract org.rocksdb.DBOptions createDBOptions()

Creates the DBOptionsfor this pre-defined setting.

Returns:

The pre-defined options object.

createColumnOptions

public abstract org.rocksdb.ColumnFamilyOptions createColumnOptions()

Creates the ColumnFamilyOptionsfor this pre-defined setting.

Returns:

The pre-defined options object.