org.apache.flink.runtime.jobmanager

Class ZooKeeperJobGraphStore

java.lang.Object

org.apache.flink.runtime.jobmanager.ZooKeeperJobGraphStore

All Implemented Interfaces:

JobGraphStore, JobGraphWriter

public class ZooKeeperJobGraphStore
extends Object
implements JobGraphStore

JobGraph instances for JobManagers running in HighAvailabilityMode.ZOOKEEPER.

Each job graph creates ZNode:

 +----O /flink/jobgraphs/<job-id> 1 [persistent]
 .
 .
 .
 +----O /flink/jobgraphs/<job-id> N [persistent]
 

The root path is watched to detect concurrent modifications in corner situations where multiple instances operate concurrently. The job manager acts as a JobGraphListener to react to such situations.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.flink.runtime.jobmanager.JobGraphStore

JobGraphStore.JobGraphListener

Constructor Summary

Constructors

Constructor and Description
ZooKeeperJobGraphStore(String zooKeeperFullBasePath, ZooKeeperStateHandleStore<JobGraph> zooKeeperStateHandleStore, org.apache.curator.framework.recipes.cache.PathChildrenCache pathCache) Submitted job graph store backed by ZooKeeper.

Method Summary

Modifier and Type	Method and Description
Collection<org.apache.flink.api.common.JobID>	getJobIds() Get all job ids of submitted job graphs to the submitted job graph store.
static String	getPathForJob(org.apache.flink.api.common.JobID jobId) Returns the JobID as a String (with leading slash).
static org.apache.flink.api.common.JobID	jobIdfromPath(String path) Returns the JobID from the given path in ZooKeeper.
void	putJobGraph(JobGraph jobGraph) Adds the JobGraph instance.
JobGraph	recoverJobGraph(org.apache.flink.api.common.JobID jobId) Returns the JobGraph with the given JobID or null if no job was registered.
void	releaseJobGraph(org.apache.flink.api.common.JobID jobId) Releases the locks on the specified JobGraph.
void	removeJobGraph(org.apache.flink.api.common.JobID jobId) Removes the JobGraph with the given JobID if it exists.
void	start(JobGraphStore.JobGraphListener jobGraphListener) Starts the JobGraphStore service.
void	stop() Stops the JobGraphStore service.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ZooKeeperJobGraphStore

public ZooKeeperJobGraphStore(String zooKeeperFullBasePath,
                              ZooKeeperStateHandleStore<JobGraph> zooKeeperStateHandleStore,
                              org.apache.curator.framework.recipes.cache.PathChildrenCache pathCache)

Submitted job graph store backed by ZooKeeper.

Parameters:

zooKeeperFullBasePath - ZooKeeper path for current job graphs

zooKeeperStateHandleStore - State storage used to persist the submitted jobs

Method Detail

start

public void start(JobGraphStore.JobGraphListener jobGraphListener)
           throws Exception

Description copied from interface: JobGraphStore

Starts the JobGraphStore service.

Specified by:

start in interface JobGraphStore

Throws:

Exception

stop

public void stop()
          throws Exception

Description copied from interface: JobGraphStore

Stops the JobGraphStore service.

Specified by:

stop in interface JobGraphStore

Throws:

Exception

recoverJobGraph

@Nullable
public JobGraph recoverJobGraph(org.apache.flink.api.common.JobID jobId)
                                   throws Exception

Description copied from interface: JobGraphStore

Returns the JobGraph with the given JobID or null if no job was registered.

Specified by:

recoverJobGraph in interface JobGraphStore

Throws:

Exception

putJobGraph

public void putJobGraph(JobGraph jobGraph)
                 throws Exception

Description copied from interface: JobGraphWriter

Adds the JobGraph instance.

If a job graph with the same JobID exists, it is replaced.

Specified by:

putJobGraph in interface JobGraphWriter

Throws:

Exception

removeJobGraph

public void removeJobGraph(org.apache.flink.api.common.JobID jobId)
                    throws Exception

Description copied from interface: JobGraphWriter

Removes the JobGraph with the given JobID if it exists.

Specified by:

removeJobGraph in interface JobGraphWriter

Throws:

Exception

releaseJobGraph

public void releaseJobGraph(org.apache.flink.api.common.JobID jobId)
                     throws Exception

Description copied from interface: JobGraphWriter

Releases the locks on the specified JobGraph. Releasing the locks allows that another instance can delete the job from the JobGraphStore.

Specified by:

releaseJobGraph in interface JobGraphWriter

Parameters:

jobId - specifying the job to release the locks for

Throws:

Exception - if the locks cannot be released

getJobIds

public Collection<org.apache.flink.api.common.JobID> getJobIds()
                                                        throws Exception

Description copied from interface: JobGraphStore

Get all job ids of submitted job graphs to the submitted job graph store.

Specified by:

getJobIds in interface JobGraphStore

Returns:

Collection of submitted job ids

Throws:

Exception - if the operation fails

getPathForJob

public static String getPathForJob(org.apache.flink.api.common.JobID jobId)

Returns the JobID as a String (with leading slash).

jobIdfromPath

public static org.apache.flink.api.common.JobID jobIdfromPath(String path)

Returns the JobID from the given path in ZooKeeper.

Parameters:

path - in ZooKeeper

Returns:

JobID associated with the given path