org.apache.flink.runtime.state.heap

Class HeapKeyedStateBackend<K>

java.lang.Object

org.apache.flink.runtime.state.AbstractKeyedStateBackend<K>

org.apache.flink.runtime.state.heap.HeapKeyedStateBackend<K>

Type Parameters:

K - The key by which state is keyed.

All Implemented Interfaces:

Closeable, AutoCloseable, CheckpointListener, InternalKeyContext<K>, KeyedStateBackend<K>, Snapshotable<SnapshotResult<KeyedStateHandle>,Collection<KeyedStateHandle>>, org.apache.flink.util.Disposable

public class HeapKeyedStateBackend<K>
extends AbstractKeyedStateBackend<K>

A AbstractKeyedStateBackend that keeps state on the Java Heap and will serialize state to streams provided by a CheckpointStreamFactory upon checkpointing.

Field Summary

Fields inherited from class org.apache.flink.runtime.state.AbstractKeyedStateBackend

cancelStreamRegistry, currentKey, keyGroupCompressionDecorator, keyGroupRange, keySerializer, keyValueStatesByName, kvStateRegistry, numberOfKeyGroups, userCodeClassLoader

Constructor Summary

Constructors

Constructor and Description
HeapKeyedStateBackend(TaskKvStateRegistry kvStateRegistry, org.apache.flink.api.common.typeutils.TypeSerializer<K> keySerializer, ClassLoader userCodeClassLoader, int numberOfKeyGroups, KeyGroupRange keyGroupRange, boolean asynchronousSnapshots, org.apache.flink.api.common.ExecutionConfig executionConfig, LocalRecoveryConfig localRecoveryConfig)

Method Summary

Modifier and Type	Method and Description
<N,S extends org.apache.flink.api.common.state.State,T> void	applyToAllKeys(N namespace, org.apache.flink.api.common.typeutils.TypeSerializer<N> namespaceSerializer, org.apache.flink.api.common.state.StateDescriptor<S,T> stateDescriptor, KeyedStateFunction<K,S> function) Applies the provided KeyedStateFunction to the state with the provided StateDescriptor of all the currently active keys.
<N,T,ACC,R> InternalAggregatingState<K,N,T,ACC,R>	createAggregatingState(org.apache.flink.api.common.typeutils.TypeSerializer<N> namespaceSerializer, org.apache.flink.api.common.state.AggregatingStateDescriptor<T,ACC,R> stateDesc) Creates and returns a new AggregatingState.
<N,T,ACC> InternalFoldingState<K,N,T,ACC>	createFoldingState(org.apache.flink.api.common.typeutils.TypeSerializer<N> namespaceSerializer, org.apache.flink.api.common.state.FoldingStateDescriptor<T,ACC> stateDesc) Creates and returns a new FoldingState.
<N,T> InternalListState<K,N,T>	createListState(org.apache.flink.api.common.typeutils.TypeSerializer<N> namespaceSerializer, org.apache.flink.api.common.state.ListStateDescriptor<T> stateDesc) Creates and returns a new ListState.
protected <N,UK,UV> InternalMapState<K,N,UK,UV>	createMapState(org.apache.flink.api.common.typeutils.TypeSerializer<N> namespaceSerializer, org.apache.flink.api.common.state.MapStateDescriptor<UK,UV> stateDesc) Creates and returns a new MapState.
<N,T> InternalReducingState<K,N,T>	createReducingState(org.apache.flink.api.common.typeutils.TypeSerializer<N> namespaceSerializer, org.apache.flink.api.common.state.ReducingStateDescriptor<T> stateDesc) Creates and returns a new ReducingState.
<N,V> InternalValueState<K,N,V>	createValueState(org.apache.flink.api.common.typeutils.TypeSerializer<N> namespaceSerializer, org.apache.flink.api.common.state.ValueStateDescriptor<V> stateDesc) Creates and returns a new ValueState.
<N> java.util.stream.Stream<K>	getKeys(String state, N namespace)
LocalRecoveryConfig	getLocalRecoveryConfig()
void	notifyCheckpointComplete(long checkpointId) This method is called as a notification once a distributed checkpoint has been completed.
int	numStateEntries() Returns the total number of state entries across all keys/namespaces.
int	numStateEntries(Object namespace) Returns the total number of state entries across all keys for the given namespace.
void	restore(Collection<KeyedStateHandle> restoredState) Restores state that was previously snapshotted from the provided parameters.
RunnableFuture<SnapshotResult<KeyedStateHandle>>	snapshot(long checkpointId, long timestamp, CheckpointStreamFactory streamFactory, CheckpointOptions checkpointOptions) Operation that writes a snapshot into a stream that is provided by the given CheckpointStreamFactory and returns a @RunnableFuture that gives a state handle to the snapshot.
boolean	supportsAsynchronousSnapshots()
String	toString()

Methods inherited from class org.apache.flink.runtime.state.AbstractKeyedStateBackend

close, dispose, getCurrentKey, getCurrentKeyGroupIndex, getKeyGroupCompressionDecorator, getKeyGroupRange, getKeySerializer, getNumberOfKeyGroups, getOrCreateKeyedState, getPartitionedState, setCurrentKey

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

HeapKeyedStateBackend

public HeapKeyedStateBackend(TaskKvStateRegistry kvStateRegistry,
                             org.apache.flink.api.common.typeutils.TypeSerializer<K> keySerializer,
                             ClassLoader userCodeClassLoader,
                             int numberOfKeyGroups,
                             KeyGroupRange keyGroupRange,
                             boolean asynchronousSnapshots,
                             org.apache.flink.api.common.ExecutionConfig executionConfig,
                             LocalRecoveryConfig localRecoveryConfig)

Method Detail

getKeys

public <N> java.util.stream.Stream<K> getKeys(String state,
                                              N namespace)

Parameters:

state - State variable for which existing keys will be returned.

namespace - Namespace for which existing keys will be returned.

Returns:

A stream of all keys for the given state and namespace. Modifications to the state during iterating over it keys are not supported.

createValueState

public <N,V> InternalValueState<K,N,V> createValueState(org.apache.flink.api.common.typeutils.TypeSerializer<N> namespaceSerializer,
                                                        org.apache.flink.api.common.state.ValueStateDescriptor<V> stateDesc)
                                                 throws Exception

Description copied from class: AbstractKeyedStateBackend

Creates and returns a new ValueState.

Specified by:

createValueState in class AbstractKeyedStateBackend<K>

Type Parameters:

N - The type of the namespace.

V - The type of the value that the ValueState can store.

Parameters:

namespaceSerializer - TypeSerializer for the state namespace.

stateDesc - The StateDescriptor that contains the name of the state.

Throws:

Exception

createListState

public <N,T> InternalListState<K,N,T> createListState(org.apache.flink.api.common.typeutils.TypeSerializer<N> namespaceSerializer,
                                                      org.apache.flink.api.common.state.ListStateDescriptor<T> stateDesc)
                                               throws Exception

Description copied from class: AbstractKeyedStateBackend

Creates and returns a new ListState.

Specified by:

createListState in class AbstractKeyedStateBackend<K>

Type Parameters:

N - The type of the namespace.

T - The type of the values that the ListState can store.

Parameters:

namespaceSerializer - TypeSerializer for the state namespace.

stateDesc - The StateDescriptor that contains the name of the state.

Throws:

Exception

createReducingState

public <N,T> InternalReducingState<K,N,T> createReducingState(org.apache.flink.api.common.typeutils.TypeSerializer<N> namespaceSerializer,
                                                              org.apache.flink.api.common.state.ReducingStateDescriptor<T> stateDesc)
                                                       throws Exception

Description copied from class: AbstractKeyedStateBackend

Creates and returns a new ReducingState.

Specified by:

createReducingState in class AbstractKeyedStateBackend<K>

Type Parameters:

N - The type of the namespace.

T - The type of the values that the ListState can store.

Parameters:

namespaceSerializer - TypeSerializer for the state namespace.

stateDesc - The StateDescriptor that contains the name of the state.

Throws:

Exception

createAggregatingState

public <N,T,ACC,R> InternalAggregatingState<K,N,T,ACC,R> createAggregatingState(org.apache.flink.api.common.typeutils.TypeSerializer<N> namespaceSerializer,
                                                                                org.apache.flink.api.common.state.AggregatingStateDescriptor<T,ACC,R> stateDesc)
                                                                         throws Exception

Description copied from class: AbstractKeyedStateBackend

Creates and returns a new AggregatingState.

Specified by:

createAggregatingState in class AbstractKeyedStateBackend<K>

Type Parameters:

N - The type of the namespace.

T - The type of the values that the ListState can store.

Parameters:

namespaceSerializer - TypeSerializer for the state namespace.

stateDesc - The StateDescriptor that contains the name of the state.

Throws:

Exception

createFoldingState

public <N,T,ACC> InternalFoldingState<K,N,T,ACC> createFoldingState(org.apache.flink.api.common.typeutils.TypeSerializer<N> namespaceSerializer,
                                                                    org.apache.flink.api.common.state.FoldingStateDescriptor<T,ACC> stateDesc)
                                                             throws Exception

Description copied from class: AbstractKeyedStateBackend

Creates and returns a new FoldingState.

Specified by:

createFoldingState in class AbstractKeyedStateBackend<K>

Type Parameters:

N - The type of the namespace.

T - Type of the values folded into the state

ACC - Type of the value in the state

Parameters:

namespaceSerializer - TypeSerializer for the state namespace.

stateDesc - The StateDescriptor that contains the name of the state.

Throws:

Exception

createMapState

protected <N,UK,UV> InternalMapState<K,N,UK,UV> createMapState(org.apache.flink.api.common.typeutils.TypeSerializer<N> namespaceSerializer,
                                                               org.apache.flink.api.common.state.MapStateDescriptor<UK,UV> stateDesc)
                                                        throws Exception

Description copied from class: AbstractKeyedStateBackend

Creates and returns a new MapState.

Specified by:

createMapState in class AbstractKeyedStateBackend<K>

Type Parameters:

N - The type of the namespace.

UK - Type of the keys in the state

UV - Type of the values in the state *

Parameters:

namespaceSerializer - TypeSerializer for the state namespace.

stateDesc - The StateDescriptor that contains the name of the state.

Throws:

Exception

snapshot

public RunnableFuture<SnapshotResult<KeyedStateHandle>> snapshot(long checkpointId,
                                                                 long timestamp,
                                                                 CheckpointStreamFactory streamFactory,
                                                                 CheckpointOptions checkpointOptions)

Description copied from interface: Snapshotable

Operation that writes a snapshot into a stream that is provided by the given CheckpointStreamFactory and returns a @RunnableFuture that gives a state handle to the snapshot. It is up to the implementation if the operation is performed synchronous or asynchronous. In the later case, the returned Runnable must be executed first before obtaining the handle.

Parameters:

checkpointId - The ID of the checkpoint.

timestamp - The timestamp of the checkpoint.

streamFactory - The factory that we can use for writing our state to streams.

checkpointOptions - Options for how to perform this checkpoint.

Returns:

A runnable future that will yield a StateObject.

restore

public void restore(Collection<KeyedStateHandle> restoredState)
             throws Exception

Description copied from interface: Snapshotable

Restores state that was previously snapshotted from the provided parameters. Typically the parameters are state handles from which the old state is read.

Parameters:

restoredState - the old state to restore.

Throws:

Exception

notifyCheckpointComplete

public void notifyCheckpointComplete(long checkpointId)

Description copied from interface: CheckpointListener

This method is called as a notification once a distributed checkpoint has been completed. Note that any exception during this method will not cause the checkpoint to fail any more.

Parameters:

checkpointId - The ID of the checkpoint that has been completed.

applyToAllKeys

public <N,S extends org.apache.flink.api.common.state.State,T> void applyToAllKeys(N namespace,
                                                                                   org.apache.flink.api.common.typeutils.TypeSerializer<N> namespaceSerializer,
                                                                                   org.apache.flink.api.common.state.StateDescriptor<S,T> stateDescriptor,
                                                                                   KeyedStateFunction<K,S> function)
                                                                            throws Exception

Description copied from interface: KeyedStateBackend

Applies the provided KeyedStateFunction to the state with the provided StateDescriptor of all the currently active keys.

Specified by:

applyToAllKeys in interface KeyedStateBackend<K>

Overrides:

applyToAllKeys in class AbstractKeyedStateBackend<K>

Type Parameters:

N - The type of the namespace.

S - The type of the state.

Parameters:

namespace - the namespace of the state.

namespaceSerializer - the serializer for the namespace.

stateDescriptor - the descriptor of the state to which the function is going to be applied.

function - the function to be applied to the keyed state.

Throws:

Exception

See Also:

KeyedStateBackend

toString

public String toString()

Overrides:

toString in class Object

numStateEntries

@VisibleForTesting
public int numStateEntries()

Returns the total number of state entries across all keys/namespaces.

Specified by:

numStateEntries in class AbstractKeyedStateBackend<K>

numStateEntries

@VisibleForTesting
public int numStateEntries(Object namespace)

Returns the total number of state entries across all keys for the given namespace.

supportsAsynchronousSnapshots

public boolean supportsAsynchronousSnapshots()

Overrides:

supportsAsynchronousSnapshots in class AbstractKeyedStateBackend<K>

getLocalRecoveryConfig

@VisibleForTesting
public LocalRecoveryConfig getLocalRecoveryConfig()