org.apache.flink.kubernetes.highavailability

Class KubernetesHaServices

java.lang.Object

org.apache.flink.runtime.highavailability.AbstractHaServices

org.apache.flink.kubernetes.highavailability.KubernetesHaServices

All Implemented Interfaces:

AutoCloseable, org.apache.flink.runtime.highavailability.ClientHighAvailabilityServices, org.apache.flink.runtime.highavailability.HighAvailabilityServices

public class KubernetesHaServices
extends org.apache.flink.runtime.highavailability.AbstractHaServices

An implementation of the AbstractHaServices using Kubernetes.

All the HA information relevant for a specific component will be stored in a single ConfigMap. For example, the Dispatcher's ConfigMap would then contain the current leader, the running jobs and the pointers to the persisted JobGraphs. The JobManager's ConfigMap would then contain the current leader, the pointers to the checkpoints and the checkpoint ID counter.

The ConfigMap name will be created with the pattern "{clusterId}-{componentName}-leader". Given that the cluster id is configured to "k8s-ha-app1", then we could get the following ConfigMap names. e.g. k8s-ha-app1-restserver-leader, k8s-ha-app1-00000000000000000000000000000000-jobmanager-leader

Note that underline("_") is not allowed in Kubernetes ConfigMap name.

Field Summary

Fields inherited from class org.apache.flink.runtime.highavailability.AbstractHaServices

configuration, ioExecutor, logger

Fields inherited from interface org.apache.flink.runtime.highavailability.HighAvailabilityServices

DEFAULT_JOB_ID, DEFAULT_LEADER_ID

Method Summary

Modifier and Type	Method and Description
org.apache.flink.runtime.checkpoint.CheckpointRecoveryFactory	createCheckpointRecoveryFactory()
org.apache.flink.runtime.jobmanager.JobGraphStore	createJobGraphStore()
org.apache.flink.runtime.leaderelection.LeaderElectionService	createLeaderElectionService(String leaderName)
org.apache.flink.runtime.leaderretrieval.LeaderRetrievalService	createLeaderRetrievalService(String leaderName)
org.apache.flink.runtime.highavailability.RunningJobsRegistry	createRunningJobsRegistry()
protected String	getLeaderPathForDispatcher()
String	getLeaderPathForJobManager(org.apache.flink.api.common.JobID jobID)
protected String	getLeaderPathForResourceManager()
protected String	getLeaderPathForRestServer()
void	internalCleanup()
void	internalCleanupJobData(org.apache.flink.api.common.JobID jobID)
void	internalClose()

Methods inherited from class org.apache.flink.runtime.highavailability.AbstractHaServices

cleanupJobData, close, closeAndCleanupAllData, createBlobStore, getCheckpointRecoveryFactory, getClusterRestEndpointLeaderElectionService, getClusterRestEndpointLeaderRetriever, getDispatcherLeaderElectionService, getDispatcherLeaderRetriever, getJobGraphStore, getJobManagerLeaderElectionService, getJobManagerLeaderRetriever, getJobManagerLeaderRetriever, getResourceManagerLeaderElectionService, getResourceManagerLeaderRetriever, getRunningJobsRegistry

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.runtime.highavailability.HighAvailabilityServices

getWebMonitorLeaderElectionService, getWebMonitorLeaderRetriever

Method Detail

createLeaderElectionService

public org.apache.flink.runtime.leaderelection.LeaderElectionService createLeaderElectionService(String leaderName)

Specified by:

createLeaderElectionService in class org.apache.flink.runtime.highavailability.AbstractHaServices

createLeaderRetrievalService

public org.apache.flink.runtime.leaderretrieval.LeaderRetrievalService createLeaderRetrievalService(String leaderName)

Specified by:

createLeaderRetrievalService in class org.apache.flink.runtime.highavailability.AbstractHaServices

createCheckpointRecoveryFactory

public org.apache.flink.runtime.checkpoint.CheckpointRecoveryFactory createCheckpointRecoveryFactory()

Specified by:

createCheckpointRecoveryFactory in class org.apache.flink.runtime.highavailability.AbstractHaServices

createJobGraphStore

public org.apache.flink.runtime.jobmanager.JobGraphStore createJobGraphStore()
                                                                      throws Exception

Specified by:

createJobGraphStore in class org.apache.flink.runtime.highavailability.AbstractHaServices

Throws:

Exception

createRunningJobsRegistry

public org.apache.flink.runtime.highavailability.RunningJobsRegistry createRunningJobsRegistry()

Specified by:

createRunningJobsRegistry in class org.apache.flink.runtime.highavailability.AbstractHaServices

internalClose

public void internalClose()

Specified by:

internalClose in class org.apache.flink.runtime.highavailability.AbstractHaServices

internalCleanup

public void internalCleanup()
                     throws Exception

Specified by:

internalCleanup in class org.apache.flink.runtime.highavailability.AbstractHaServices

Throws:

Exception

internalCleanupJobData

public void internalCleanupJobData(org.apache.flink.api.common.JobID jobID)
                            throws Exception

Specified by:

internalCleanupJobData in class org.apache.flink.runtime.highavailability.AbstractHaServices

Throws:

Exception

getLeaderPathForResourceManager

protected String getLeaderPathForResourceManager()

Specified by:

getLeaderPathForResourceManager in class org.apache.flink.runtime.highavailability.AbstractHaServices

getLeaderPathForDispatcher

protected String getLeaderPathForDispatcher()

Specified by:

getLeaderPathForDispatcher in class org.apache.flink.runtime.highavailability.AbstractHaServices

getLeaderPathForJobManager

public String getLeaderPathForJobManager(org.apache.flink.api.common.JobID jobID)

Specified by:

getLeaderPathForJobManager in class org.apache.flink.runtime.highavailability.AbstractHaServices

getLeaderPathForRestServer

protected String getLeaderPathForRestServer()

Specified by:

getLeaderPathForRestServer in class org.apache.flink.runtime.highavailability.AbstractHaServices