org.apache.flink.state.changelog

Class ChangelogKeyedStateBackend<K>

java.lang.Object

org.apache.flink.state.changelog.ChangelogKeyedStateBackend<K>

Type Parameters:

K - The key by which state is keyed.

All Implemented Interfaces:

Closeable, AutoCloseable, org.apache.flink.api.common.state.CheckpointListener, org.apache.flink.runtime.state.CheckpointableKeyedStateBackend<K>, org.apache.flink.runtime.state.KeyedStateBackend<K>, org.apache.flink.runtime.state.KeyedStateFactory, org.apache.flink.runtime.state.PriorityQueueSetFactory, org.apache.flink.runtime.state.Snapshotable<org.apache.flink.runtime.state.SnapshotResult<org.apache.flink.runtime.state.KeyedStateHandle>>, org.apache.flink.runtime.state.TestableKeyedStateBackend<K>, org.apache.flink.util.Disposable

@Internal
public class ChangelogKeyedStateBackend<K>
extends Object
implements org.apache.flink.runtime.state.CheckpointableKeyedStateBackend<K>, org.apache.flink.api.common.state.CheckpointListener, org.apache.flink.runtime.state.TestableKeyedStateBackend<K>

A KeyedStateBackend that keeps state on the underlying delegated keyed state backend as well as on the state change log.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.flink.runtime.state.KeyedStateBackend

org.apache.flink.runtime.state.KeyedStateBackend.KeySelectionListener<K>

Constructor Summary

Constructors

Constructor and Description

ChangelogKeyedStateBackend(org.apache.flink.runtime.state.AbstractKeyedStateBackend<K> keyedStateBackend, org.apache.flink.api.common.ExecutionConfig executionConfig, org.apache.flink.runtime.state.ttl.TtlTimeProvider ttlTimeProvider, org.apache.flink.runtime.state.changelog.StateChangelogWriter<org.apache.flink.runtime.state.changelog.ChangelogStateHandle> stateChangelogWriter, Collection<org.apache.flink.runtime.state.changelog.ChangelogStateBackendHandle> initialState, org.apache.flink.api.common.operators.MailboxExecutor mainMailboxExecutor, ExecutorService asyncOperationsThreadPool)

Method Summary

Modifier and Type	Method and Description
<N,S extends org.apache.flink.api.common.state.State,T> void	applyToAllKeys(N namespace, org.apache.flink.api.common.typeutils.TypeSerializer<N> namespaceSerializer, org.apache.flink.api.common.state.StateDescriptor<S,T> stateDescriptor, org.apache.flink.runtime.state.KeyedStateFunction<K,S> function)
void	close()
<T extends org.apache.flink.runtime.state.heap.HeapPriorityQueueElement & org.apache.flink.runtime.state.PriorityComparable<? super T> & org.apache.flink.runtime.state.Keyed<?>> org.apache.flink.runtime.state.KeyGroupedInternalPriorityQueue<T>	create(String stateName, org.apache.flink.api.common.typeutils.TypeSerializer<T> byteOrderedElementSerializer)
<N,SV,SEV,S extends org.apache.flink.api.common.state.State,IS extends S> IS	createInternalState(org.apache.flink.api.common.typeutils.TypeSerializer<N> namespaceSerializer, org.apache.flink.api.common.state.StateDescriptor<S,SV> stateDesc, org.apache.flink.runtime.state.StateSnapshotTransformer.StateSnapshotTransformFactory<SEV> snapshotTransformFactory)
boolean	deregisterKeySelectionListener(org.apache.flink.runtime.state.KeyedStateBackend.KeySelectionListener<K> listener)
void	dispose()
K	getCurrentKey()
org.apache.flink.runtime.state.KeyedStateBackend<K>	getDelegatedKeyedStateBackend(boolean recursive)
ChangelogState	getExistingStateForRecovery(String name, org.apache.flink.runtime.state.metainfo.StateMetaInfoSnapshot.BackendStateType type)
org.apache.flink.runtime.state.KeyGroupRange	getKeyGroupRange()
<N> java.util.stream.Stream<K>	getKeys(String state, N namespace)
<N> java.util.stream.Stream<org.apache.flink.api.java.tuple.Tuple2<K,N>>	getKeysAndNamespaces(String state)
org.apache.flink.api.common.typeutils.TypeSerializer<K>	getKeySerializer()
<N,S extends org.apache.flink.api.common.state.State,T> S	getOrCreateKeyedState(org.apache.flink.api.common.typeutils.TypeSerializer<N> namespaceSerializer, org.apache.flink.api.common.state.StateDescriptor<S,T> stateDescriptor)
<N,S extends org.apache.flink.api.common.state.State> S	getPartitionedState(N namespace, org.apache.flink.api.common.typeutils.TypeSerializer<N> namespaceSerializer, org.apache.flink.api.common.state.StateDescriptor<S,?> stateDescriptor)
boolean	isStateImmutableInStateBackend(org.apache.flink.runtime.checkpoint.CheckpointType checkpointOptions)
void	notifyCheckpointAborted(long checkpointId)
void	notifyCheckpointComplete(long checkpointId)
int	numKeyValueStateEntries()
void	registerKeySelectionListener(org.apache.flink.runtime.state.KeyedStateBackend.KeySelectionListener<K> listener)
org.apache.flink.runtime.state.SavepointResources<K>	savepoint()
void	setCurrentKey(K newKey)
RunnableFuture<org.apache.flink.runtime.state.SnapshotResult<org.apache.flink.runtime.state.KeyedStateHandle>>	snapshot(long checkpointId, long timestamp, org.apache.flink.runtime.state.CheckpointStreamFactory streamFactory, org.apache.flink.runtime.checkpoint.CheckpointOptions checkpointOptions)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.runtime.state.KeyedStateFactory

createInternalState

Constructor Detail

ChangelogKeyedStateBackend

public ChangelogKeyedStateBackend(org.apache.flink.runtime.state.AbstractKeyedStateBackend<K> keyedStateBackend,
                                  org.apache.flink.api.common.ExecutionConfig executionConfig,
                                  org.apache.flink.runtime.state.ttl.TtlTimeProvider ttlTimeProvider,
                                  org.apache.flink.runtime.state.changelog.StateChangelogWriter<org.apache.flink.runtime.state.changelog.ChangelogStateHandle> stateChangelogWriter,
                                  Collection<org.apache.flink.runtime.state.changelog.ChangelogStateBackendHandle> initialState,
                                  org.apache.flink.api.common.operators.MailboxExecutor mainMailboxExecutor,
                                  ExecutorService asyncOperationsThreadPool)

Method Detail

getKeyGroupRange

public org.apache.flink.runtime.state.KeyGroupRange getKeyGroupRange()

Specified by:

getKeyGroupRange in interface org.apache.flink.runtime.state.CheckpointableKeyedStateBackend<K>

close

public void close()
           throws IOException

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Throws:

IOException

setCurrentKey

public void setCurrentKey(K newKey)

Specified by:

setCurrentKey in interface org.apache.flink.runtime.state.KeyedStateBackend<K>

getCurrentKey

public K getCurrentKey()

Specified by:

getCurrentKey in interface org.apache.flink.runtime.state.KeyedStateBackend<K>

getKeySerializer

public org.apache.flink.api.common.typeutils.TypeSerializer<K> getKeySerializer()

Specified by:

getKeySerializer in interface org.apache.flink.runtime.state.KeyedStateBackend<K>

getKeys

public <N> java.util.stream.Stream<K> getKeys(String state,
                                              N namespace)

Specified by:

getKeys in interface org.apache.flink.runtime.state.KeyedStateBackend<K>

getKeysAndNamespaces

public <N> java.util.stream.Stream<org.apache.flink.api.java.tuple.Tuple2<K,N>> getKeysAndNamespaces(String state)

Specified by:

getKeysAndNamespaces in interface org.apache.flink.runtime.state.KeyedStateBackend<K>

dispose

public void dispose()

Specified by:

dispose in interface org.apache.flink.runtime.state.KeyedStateBackend<K>

Specified by:

dispose in interface org.apache.flink.util.Disposable

registerKeySelectionListener

public void registerKeySelectionListener(org.apache.flink.runtime.state.KeyedStateBackend.KeySelectionListener<K> listener)

Specified by:

registerKeySelectionListener in interface org.apache.flink.runtime.state.KeyedStateBackend<K>

deregisterKeySelectionListener

public boolean deregisterKeySelectionListener(org.apache.flink.runtime.state.KeyedStateBackend.KeySelectionListener<K> listener)

Specified by:

deregisterKeySelectionListener in interface org.apache.flink.runtime.state.KeyedStateBackend<K>

applyToAllKeys

public <N,S extends org.apache.flink.api.common.state.State,T> void applyToAllKeys(N namespace,
                                                                                   org.apache.flink.api.common.typeutils.TypeSerializer<N> namespaceSerializer,
                                                                                   org.apache.flink.api.common.state.StateDescriptor<S,T> stateDescriptor,
                                                                                   org.apache.flink.runtime.state.KeyedStateFunction<K,S> function)
                                                                            throws Exception

Specified by:

applyToAllKeys in interface org.apache.flink.runtime.state.KeyedStateBackend<K>

Throws:

Exception

getPartitionedState

public <N,S extends org.apache.flink.api.common.state.State> S getPartitionedState(N namespace,
                                                                                   org.apache.flink.api.common.typeutils.TypeSerializer<N> namespaceSerializer,
                                                                                   org.apache.flink.api.common.state.StateDescriptor<S,?> stateDescriptor)
                                                                            throws Exception

Specified by:

getPartitionedState in interface org.apache.flink.runtime.state.KeyedStateBackend<K>

Throws:

Exception

snapshot

@Nonnull
public RunnableFuture<org.apache.flink.runtime.state.SnapshotResult<org.apache.flink.runtime.state.KeyedStateHandle>> snapshot(long checkpointId,
                                                                                                                                        long timestamp,
                                                                                                                                        @Nonnull
                                                                                                                                        org.apache.flink.runtime.state.CheckpointStreamFactory streamFactory,
                                                                                                                                        @Nonnull
                                                                                                                                        org.apache.flink.runtime.checkpoint.CheckpointOptions checkpointOptions)
                                                                                                                                 throws Exception

Specified by:

snapshot in interface org.apache.flink.runtime.state.Snapshotable<org.apache.flink.runtime.state.SnapshotResult<org.apache.flink.runtime.state.KeyedStateHandle>>

Throws:

Exception

create

@Nonnull
public <T extends org.apache.flink.runtime.state.heap.HeapPriorityQueueElement & org.apache.flink.runtime.state.PriorityComparable<? super T> & org.apache.flink.runtime.state.Keyed<?>> org.apache.flink.runtime.state.KeyGroupedInternalPriorityQueue<T> create(@Nonnull
                                                                                                                                                                                                                                                                           String stateName,
                                                                                                                                                                                                                                                                           @Nonnull
                                                                                                                                                                                                                                                                           org.apache.flink.api.common.typeutils.TypeSerializer<T> byteOrderedElementSerializer)

Specified by:

create in interface org.apache.flink.runtime.state.PriorityQueueSetFactory

numKeyValueStateEntries

@VisibleForTesting
public int numKeyValueStateEntries()

Specified by:

numKeyValueStateEntries in interface org.apache.flink.runtime.state.TestableKeyedStateBackend<K>

isStateImmutableInStateBackend

public boolean isStateImmutableInStateBackend(org.apache.flink.runtime.checkpoint.CheckpointType checkpointOptions)

Specified by:

isStateImmutableInStateBackend in interface org.apache.flink.runtime.state.KeyedStateBackend<K>

savepoint

@Nonnull
public org.apache.flink.runtime.state.SavepointResources<K> savepoint()
                                                                        throws Exception

Specified by:

savepoint in interface org.apache.flink.runtime.state.CheckpointableKeyedStateBackend<K>

Throws:

Exception

notifyCheckpointComplete

public void notifyCheckpointComplete(long checkpointId)
                              throws Exception

Specified by:

notifyCheckpointComplete in interface org.apache.flink.api.common.state.CheckpointListener

Throws:

Exception

notifyCheckpointAborted

public void notifyCheckpointAborted(long checkpointId)
                             throws Exception

Specified by:

notifyCheckpointAborted in interface org.apache.flink.api.common.state.CheckpointListener

Throws:

Exception

getOrCreateKeyedState

public <N,S extends org.apache.flink.api.common.state.State,T> S getOrCreateKeyedState(org.apache.flink.api.common.typeutils.TypeSerializer<N> namespaceSerializer,
                                                                                       org.apache.flink.api.common.state.StateDescriptor<S,T> stateDescriptor)
                                                                                throws Exception

Specified by:

getOrCreateKeyedState in interface org.apache.flink.runtime.state.KeyedStateBackend<K>

Throws:

Exception

createInternalState

@Nonnull
public <N,SV,SEV,S extends org.apache.flink.api.common.state.State,IS extends S> IS createInternalState(@Nonnull
                                                                                                                 org.apache.flink.api.common.typeutils.TypeSerializer<N> namespaceSerializer,
                                                                                                                 @Nonnull
                                                                                                                 org.apache.flink.api.common.state.StateDescriptor<S,SV> stateDesc,
                                                                                                                 @Nonnull
                                                                                                                 org.apache.flink.runtime.state.StateSnapshotTransformer.StateSnapshotTransformFactory<SEV> snapshotTransformFactory)
                                                                                                          throws Exception

Specified by:

createInternalState in interface org.apache.flink.runtime.state.KeyedStateFactory

Throws:

Exception

getDelegatedKeyedStateBackend

public org.apache.flink.runtime.state.KeyedStateBackend<K> getDelegatedKeyedStateBackend(boolean recursive)

Specified by:

getDelegatedKeyedStateBackend in interface org.apache.flink.runtime.state.TestableKeyedStateBackend<K>

getExistingStateForRecovery

public ChangelogState getExistingStateForRecovery(String name,
                                                  org.apache.flink.runtime.state.metainfo.StateMetaInfoSnapshot.BackendStateType type)
                                           throws NoSuchElementException,
                                                  UnsupportedOperationException

Parameters:

name - state name

type - state type (the only supported type currently are: key value, priority queue)

Returns:

an existing state, i.e. the one that was already created. The returned state will not apply TTL to the passed values, regardless of the TTL settings. This prevents double applying of TTL (recovered values are TTL values if TTL was enabled). The state will, however, use TTL serializer if TTL is enabled. WARN: only valid during the recovery.

Throws:

NoSuchElementException - if the state wasn't created

UnsupportedOperationException - if state type is not supported