org.apache.flink.runtime.state.heap

Class HeapKeyedStateBackend<K>

java.lang.Object

org.apache.flink.runtime.state.AbstractKeyedStateBackend<K>

org.apache.flink.runtime.state.heap.HeapKeyedStateBackend<K>

Type Parameters:

K - The key by which state is keyed.

All Implemented Interfaces:

Closeable, AutoCloseable, CheckpointListener, KeyedStateBackend<K>, KeyedStateFactory, PriorityQueueSetFactory, SnapshotStrategy<SnapshotResult<KeyedStateHandle>>, org.apache.flink.util.Disposable

public class HeapKeyedStateBackend<K>
extends AbstractKeyedStateBackend<K>

A AbstractKeyedStateBackend that keeps state on the Java Heap and will serialize state to streams provided by a CheckpointStreamFactory upon checkpointing.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.flink.runtime.state.KeyedStateBackend

KeyedStateBackend.KeySelectionListener<K>

Field Summary

Fields inherited from class org.apache.flink.runtime.state.AbstractKeyedStateBackend

cancelStreamRegistry, keyContext, keyGroupCompressionDecorator, keyGroupRange, keySerializer, kvStateRegistry, numberOfKeyGroups, ttlTimeProvider, userCodeClassLoader

Constructor Summary

Constructors

Constructor and Description

HeapKeyedStateBackend(TaskKvStateRegistry kvStateRegistry, org.apache.flink.api.common.typeutils.TypeSerializer<K> keySerializer, ClassLoader userCodeClassLoader, org.apache.flink.api.common.ExecutionConfig executionConfig, TtlTimeProvider ttlTimeProvider, org.apache.flink.core.fs.CloseableRegistry cancelStreamRegistry, StreamCompressionDecorator keyGroupCompressionDecorator, Map<String,StateTable<K,?,?>> registeredKVStates, Map<String,HeapPriorityQueueSnapshotRestoreWrapper> registeredPQStates, LocalRecoveryConfig localRecoveryConfig, HeapPriorityQueueSetFactory priorityQueueSetFactory, org.apache.flink.runtime.state.heap.HeapSnapshotStrategy<K> snapshotStrategy, InternalKeyContext<K> keyContext)

Method Summary

Modifier and Type	Method and Description
<N,S extends org.apache.flink.api.common.state.State,T> void	applyToAllKeys(N namespace, org.apache.flink.api.common.typeutils.TypeSerializer<N> namespaceSerializer, org.apache.flink.api.common.state.StateDescriptor<S,T> stateDescriptor, KeyedStateFunction<K,S> function) Applies the provided KeyedStateFunction to the state with the provided StateDescriptor of all the currently active keys.
<T extends HeapPriorityQueueElement & PriorityComparable & Keyed> KeyGroupedInternalPriorityQueue<T>	create(String stateName, org.apache.flink.api.common.typeutils.TypeSerializer<T> byteOrderedElementSerializer) Creates a KeyGroupedInternalPriorityQueue.
<N,SV,SEV,S extends org.apache.flink.api.common.state.State,IS extends S> IS	createInternalState(org.apache.flink.api.common.typeutils.TypeSerializer<N> namespaceSerializer, org.apache.flink.api.common.state.StateDescriptor<S,SV> stateDesc, StateSnapshotTransformer.StateSnapshotTransformFactory<SEV> snapshotTransformFactory) Creates and returns a new InternalKvState.
<N> java.util.stream.Stream<K>	getKeys(String state, N namespace)
LocalRecoveryConfig	getLocalRecoveryConfig()
void	notifyCheckpointComplete(long checkpointId) This method is called as a notification once a distributed checkpoint has been completed.
int	numKeyValueStateEntries() Returns the total number of state entries across all keys/namespaces.
int	numKeyValueStateEntries(Object namespace) Returns the total number of state entries across all keys for the given namespace.
RunnableFuture<SnapshotResult<KeyedStateHandle>>	snapshot(long checkpointId, long timestamp, CheckpointStreamFactory streamFactory, CheckpointOptions checkpointOptions) Operation that writes a snapshot into a stream that is provided by the given CheckpointStreamFactory and returns a @RunnableFuture that gives a state handle to the snapshot.
boolean	supportsAsynchronousSnapshots()
String	toString()

Methods inherited from class org.apache.flink.runtime.state.AbstractKeyedStateBackend

close, deregisterKeySelectionListener, dispose, getCurrentKey, getCurrentKeyGroupIndex, getKeyGroupCompressionDecorator, getKeyGroupRange, getKeySerializer, getNumberOfKeyGroups, getOrCreateKeyedState, getPartitionedState, registerKeySelectionListener, requiresLegacySynchronousTimerSnapshots, setCurrentKey

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.flink.runtime.state.KeyedStateFactory

createInternalState

Constructor Detail

HeapKeyedStateBackend

public HeapKeyedStateBackend(TaskKvStateRegistry kvStateRegistry,
                             org.apache.flink.api.common.typeutils.TypeSerializer<K> keySerializer,
                             ClassLoader userCodeClassLoader,
                             org.apache.flink.api.common.ExecutionConfig executionConfig,
                             TtlTimeProvider ttlTimeProvider,
                             org.apache.flink.core.fs.CloseableRegistry cancelStreamRegistry,
                             StreamCompressionDecorator keyGroupCompressionDecorator,
                             Map<String,StateTable<K,?,?>> registeredKVStates,
                             Map<String,HeapPriorityQueueSnapshotRestoreWrapper> registeredPQStates,
                             LocalRecoveryConfig localRecoveryConfig,
                             HeapPriorityQueueSetFactory priorityQueueSetFactory,
                             org.apache.flink.runtime.state.heap.HeapSnapshotStrategy<K> snapshotStrategy,
                             InternalKeyContext<K> keyContext)

Method Detail

create

@Nonnull
public <T extends HeapPriorityQueueElement & PriorityComparable & Keyed> KeyGroupedInternalPriorityQueue<T> create(@Nonnull
                                                                                                                            String stateName,
                                                                                                                            @Nonnull
                                                                                                                            org.apache.flink.api.common.typeutils.TypeSerializer<T> byteOrderedElementSerializer)

Description copied from interface: PriorityQueueSetFactory

Creates a KeyGroupedInternalPriorityQueue.

Type Parameters:

T - type of the stored elements.

Parameters:

stateName - unique name for associated with this queue.

byteOrderedElementSerializer - a serializer that with a format that is lexicographically ordered in alignment with elementPriorityComparator.

Returns:

the queue with the specified unique name.

getKeys

public <N> java.util.stream.Stream<K> getKeys(String state,
                                              N namespace)

Parameters:

state - State variable for which existing keys will be returned.

namespace - Namespace for which existing keys will be returned.

Returns:

A stream of all keys for the given state and namespace. Modifications to the state during iterating over it keys are not supported.

createInternalState

@Nonnull
public <N,SV,SEV,S extends org.apache.flink.api.common.state.State,IS extends S> IS createInternalState(@Nonnull
                                                                                                                 org.apache.flink.api.common.typeutils.TypeSerializer<N> namespaceSerializer,
                                                                                                                 @Nonnull
                                                                                                                 org.apache.flink.api.common.state.StateDescriptor<S,SV> stateDesc,
                                                                                                                 @Nonnull
                                                                                                                 StateSnapshotTransformer.StateSnapshotTransformFactory<SEV> snapshotTransformFactory)
                                                                                                          throws Exception

Description copied from interface: KeyedStateFactory

Creates and returns a new InternalKvState.

Type Parameters:

N - The type of the namespace.

SV - The type of the stored state value.

SEV - The type of the stored state value or entry for collection types (list or map).

S - The type of the public API state.

IS - The type of internal state.

Parameters:

namespaceSerializer - TypeSerializer for the state namespace.

stateDesc - The StateDescriptor that contains the name of the state.

snapshotTransformFactory - factory of state snapshot transformer.

Throws:

Exception

snapshot

@Nonnull
public RunnableFuture<SnapshotResult<KeyedStateHandle>> snapshot(long checkpointId,
                                                                          long timestamp,
                                                                          @Nonnull
                                                                          CheckpointStreamFactory streamFactory,
                                                                          @Nonnull
                                                                          CheckpointOptions checkpointOptions)
                                                                   throws IOException

Description copied from interface: SnapshotStrategy

Operation that writes a snapshot into a stream that is provided by the given CheckpointStreamFactory and returns a @RunnableFuture that gives a state handle to the snapshot. It is up to the implementation if the operation is performed synchronous or asynchronous. In the later case, the returned Runnable must be executed first before obtaining the handle.

Parameters:

checkpointId - The ID of the checkpoint.

timestamp - The timestamp of the checkpoint.

streamFactory - The factory that we can use for writing our state to streams.

checkpointOptions - Options for how to perform this checkpoint.

Returns:

A runnable future that will yield a StateObject.

Throws:

IOException

notifyCheckpointComplete

public void notifyCheckpointComplete(long checkpointId)

Description copied from interface: CheckpointListener

This method is called as a notification once a distributed checkpoint has been completed. Note that any exception during this method will not cause the checkpoint to fail any more.

Parameters:

checkpointId - The ID of the checkpoint that has been completed.

applyToAllKeys

public <N,S extends org.apache.flink.api.common.state.State,T> void applyToAllKeys(N namespace,
                                                                                   org.apache.flink.api.common.typeutils.TypeSerializer<N> namespaceSerializer,
                                                                                   org.apache.flink.api.common.state.StateDescriptor<S,T> stateDescriptor,
                                                                                   KeyedStateFunction<K,S> function)
                                                                            throws Exception

Description copied from interface: KeyedStateBackend

Applies the provided KeyedStateFunction to the state with the provided StateDescriptor of all the currently active keys.

Specified by:

applyToAllKeys in interface KeyedStateBackend<K>

Overrides:

applyToAllKeys in class AbstractKeyedStateBackend<K>

Type Parameters:

N - The type of the namespace.

S - The type of the state.

Parameters:

namespace - the namespace of the state.

namespaceSerializer - the serializer for the namespace.

stateDescriptor - the descriptor of the state to which the function is going to be applied.

function - the function to be applied to the keyed state.

Throws:

Exception

See Also:

KeyedStateBackend

toString

public String toString()

Overrides:

toString in class Object

numKeyValueStateEntries

@VisibleForTesting
public int numKeyValueStateEntries()

Returns the total number of state entries across all keys/namespaces.

Specified by:

numKeyValueStateEntries in class AbstractKeyedStateBackend<K>

numKeyValueStateEntries

@VisibleForTesting
public int numKeyValueStateEntries(Object namespace)

Returns the total number of state entries across all keys for the given namespace.

supportsAsynchronousSnapshots

public boolean supportsAsynchronousSnapshots()

Overrides:

supportsAsynchronousSnapshots in class AbstractKeyedStateBackend<K>

getLocalRecoveryConfig

@VisibleForTesting
public LocalRecoveryConfig getLocalRecoveryConfig()