org.apache.flink.contrib.streaming.state.snapshot

Class RocksDBSnapshotStrategyBase<K>

java.lang.Object

org.apache.flink.runtime.state.AbstractSnapshotStrategy<org.apache.flink.runtime.state.KeyedStateHandle>

org.apache.flink.contrib.streaming.state.snapshot.RocksDBSnapshotStrategyBase<K>

Type Parameters:

K - type of the backend keys.

All Implemented Interfaces:

org.apache.flink.runtime.state.CheckpointListener, org.apache.flink.runtime.state.SnapshotStrategy<org.apache.flink.runtime.state.SnapshotResult<org.apache.flink.runtime.state.KeyedStateHandle>>

Direct Known Subclasses:

RocksFullSnapshotStrategy, RocksIncrementalSnapshotStrategy

public abstract class RocksDBSnapshotStrategyBase<K>
extends org.apache.flink.runtime.state.AbstractSnapshotStrategy<org.apache.flink.runtime.state.KeyedStateHandle>
implements org.apache.flink.runtime.state.CheckpointListener

Abstract base class for SnapshotStrategy implementations for RocksDB state backend.

Field Summary

Fields

Modifier and Type	Field and Description
protected org.apache.flink.core.fs.CloseableRegistry	cancelStreamRegistry A CloseableRegistry that will be closed when the task is cancelled.
protected org.rocksdb.RocksDB	db RocksDB instance from the backend.
protected int	keyGroupPrefixBytes Number of bytes in the key-group prefix.
protected org.apache.flink.runtime.state.KeyGroupRange	keyGroupRange The key-group range for the task.
protected org.apache.flink.api.common.typeutils.TypeSerializer<K>	keySerializer The key serializer of the backend.
protected LinkedHashMap<String,RocksDBKeyedStateBackend.RocksDbKvStateInfo>	kvStateInformation Key/Value state meta info from the backend.
protected org.apache.flink.runtime.state.LocalRecoveryConfig	localRecoveryConfig The configuration for local recovery.
protected org.apache.flink.util.ResourceGuard	rocksDBResourceGuard Resource guard for the RocksDB instance.

Fields inherited from class org.apache.flink.runtime.state.AbstractSnapshotStrategy

description

Constructor Summary

Constructors

Constructor and Description

RocksDBSnapshotStrategyBase(String description, org.rocksdb.RocksDB db, org.apache.flink.util.ResourceGuard rocksDBResourceGuard, org.apache.flink.api.common.typeutils.TypeSerializer<K> keySerializer, LinkedHashMap<String,RocksDBKeyedStateBackend.RocksDbKvStateInfo> kvStateInformation, org.apache.flink.runtime.state.KeyGroupRange keyGroupRange, int keyGroupPrefixBytes, org.apache.flink.runtime.state.LocalRecoveryConfig localRecoveryConfig, org.apache.flink.core.fs.CloseableRegistry cancelStreamRegistry)

Method Summary

Modifier and Type	Method and Description
protected abstract RunnableFuture<org.apache.flink.runtime.state.SnapshotResult<org.apache.flink.runtime.state.KeyedStateHandle>>	doSnapshot(long checkpointId, long timestamp, org.apache.flink.runtime.state.CheckpointStreamFactory streamFactory, org.apache.flink.runtime.checkpoint.CheckpointOptions checkpointOptions) This method implements the concrete snapshot logic for a non-empty state.
RunnableFuture<org.apache.flink.runtime.state.SnapshotResult<org.apache.flink.runtime.state.KeyedStateHandle>>	snapshot(long checkpointId, long timestamp, org.apache.flink.runtime.state.CheckpointStreamFactory streamFactory, org.apache.flink.runtime.checkpoint.CheckpointOptions checkpointOptions)

Methods inherited from class org.apache.flink.runtime.state.AbstractSnapshotStrategy

logAsyncCompleted, logSyncCompleted, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.flink.runtime.state.CheckpointListener

notifyCheckpointAborted, notifyCheckpointComplete

Field Detail

db

@Nonnull
protected org.rocksdb.RocksDB db

RocksDB instance from the backend.

rocksDBResourceGuard

@Nonnull
protected final org.apache.flink.util.ResourceGuard rocksDBResourceGuard

Resource guard for the RocksDB instance.

keySerializer

@Nonnull
protected final org.apache.flink.api.common.typeutils.TypeSerializer<K> keySerializer

The key serializer of the backend.

kvStateInformation

@Nonnull
protected final LinkedHashMap<String,RocksDBKeyedStateBackend.RocksDbKvStateInfo> kvStateInformation

Key/Value state meta info from the backend.

keyGroupRange

@Nonnull
protected final org.apache.flink.runtime.state.KeyGroupRange keyGroupRange

The key-group range for the task.

keyGroupPrefixBytes

@Nonnegative
protected final int keyGroupPrefixBytes

Number of bytes in the key-group prefix.

localRecoveryConfig

@Nonnull
protected final org.apache.flink.runtime.state.LocalRecoveryConfig localRecoveryConfig

The configuration for local recovery.

cancelStreamRegistry

@Nonnull
protected final org.apache.flink.core.fs.CloseableRegistry cancelStreamRegistry

A CloseableRegistry that will be closed when the task is cancelled.

Constructor Detail

RocksDBSnapshotStrategyBase

public RocksDBSnapshotStrategyBase(@Nonnull
                                   String description,
                                   @Nonnull
                                   org.rocksdb.RocksDB db,
                                   @Nonnull
                                   org.apache.flink.util.ResourceGuard rocksDBResourceGuard,
                                   @Nonnull
                                   org.apache.flink.api.common.typeutils.TypeSerializer<K> keySerializer,
                                   @Nonnull
                                   LinkedHashMap<String,RocksDBKeyedStateBackend.RocksDbKvStateInfo> kvStateInformation,
                                   @Nonnull
                                   org.apache.flink.runtime.state.KeyGroupRange keyGroupRange,
                                   @Nonnegative
                                   int keyGroupPrefixBytes,
                                   @Nonnull
                                   org.apache.flink.runtime.state.LocalRecoveryConfig localRecoveryConfig,
                                   @Nonnull
                                   org.apache.flink.core.fs.CloseableRegistry cancelStreamRegistry)

Method Detail

snapshot

@Nonnull
public final RunnableFuture<org.apache.flink.runtime.state.SnapshotResult<org.apache.flink.runtime.state.KeyedStateHandle>> snapshot(long checkpointId,
                                                                                                                                              long timestamp,
                                                                                                                                              @Nonnull
                                                                                                                                              org.apache.flink.runtime.state.CheckpointStreamFactory streamFactory,
                                                                                                                                              @Nonnull
                                                                                                                                              org.apache.flink.runtime.checkpoint.CheckpointOptions checkpointOptions)
                                                                                                                                       throws Exception

Specified by:

snapshot in interface org.apache.flink.runtime.state.SnapshotStrategy<org.apache.flink.runtime.state.SnapshotResult<org.apache.flink.runtime.state.KeyedStateHandle>>

Throws:

Exception

doSnapshot

@Nonnull
protected abstract RunnableFuture<org.apache.flink.runtime.state.SnapshotResult<org.apache.flink.runtime.state.KeyedStateHandle>> doSnapshot(long checkpointId,
                                                                                                                                                      long timestamp,
                                                                                                                                                      org.apache.flink.runtime.state.CheckpointStreamFactory streamFactory,
                                                                                                                                                      org.apache.flink.runtime.checkpoint.CheckpointOptions checkpointOptions)
                                                                                                                                               throws Exception

This method implements the concrete snapshot logic for a non-empty state.

Throws:

Exception