org.apache.flink.runtime.jobmanager

Class ZooKeeperSubmittedJobGraphStore

java.lang.Object

org.apache.flink.runtime.jobmanager.ZooKeeperSubmittedJobGraphStore

All Implemented Interfaces:

SubmittedJobGraphStore

public class ZooKeeperSubmittedJobGraphStore
extends Object
implements SubmittedJobGraphStore

SubmittedJobGraph instances for JobManagers running in HighAvailabilityMode.ZOOKEEPER.

Each job graph creates ZNode:

 +----O /flink/jobgraphs/<job-id> 1 [persistent]
 .
 .
 .
 +----O /flink/jobgraphs/<job-id> N [persistent]
 

The root path is watched to detect concurrent modifications in corner situations where multiple instances operate concurrently. The job manager acts as a SubmittedJobGraphListener to react to such situations.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.flink.runtime.jobmanager.SubmittedJobGraphStore

SubmittedJobGraphStore.SubmittedJobGraphListener

Constructor Summary

Constructors

Constructor and Description
ZooKeeperSubmittedJobGraphStore(org.apache.curator.framework.CuratorFramework client, String currentJobsPath, RetrievableStateStorageHelper<SubmittedJobGraph> stateStorage, Executor executor) Submitted job graph store backed by ZooKeeper

Method Summary

Modifier and Type	Method and Description
Collection<org.apache.flink.api.common.JobID>	getJobIds() Get all job ids of submitted job graphs to the submitted job graph store.
static String	getPathForJob(org.apache.flink.api.common.JobID jobId) Returns the JobID as a String (with leading slash).
static org.apache.flink.api.common.JobID	jobIdfromPath(String path) Returns the JobID from the given path in ZooKeeper.
void	putJobGraph(SubmittedJobGraph jobGraph) Adds the SubmittedJobGraph instance.
SubmittedJobGraph	recoverJobGraph(org.apache.flink.api.common.JobID jobId) Returns the SubmittedJobGraph with the given JobID or null if no job was registered.
void	removeJobGraph(org.apache.flink.api.common.JobID jobId) Removes the SubmittedJobGraph with the given JobID if it exists.
void	start(SubmittedJobGraphStore.SubmittedJobGraphListener jobGraphListener) Starts the SubmittedJobGraphStore service.
void	stop() Stops the SubmittedJobGraphStore service.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ZooKeeperSubmittedJobGraphStore

public ZooKeeperSubmittedJobGraphStore(org.apache.curator.framework.CuratorFramework client,
                                       String currentJobsPath,
                                       RetrievableStateStorageHelper<SubmittedJobGraph> stateStorage,
                                       Executor executor)
                                throws Exception

Submitted job graph store backed by ZooKeeper

Parameters:

client - ZooKeeper client

currentJobsPath - ZooKeeper path for current job graphs

stateStorage - State storage used to persist the submitted jobs

executor - to give to the ZooKeeperStateHandleStore to run ZooKeeper callbacks

Throws:

Exception

Method Detail

start

public void start(SubmittedJobGraphStore.SubmittedJobGraphListener jobGraphListener)
           throws Exception

Description copied from interface: SubmittedJobGraphStore

Starts the SubmittedJobGraphStore service.

Specified by:

start in interface SubmittedJobGraphStore

Throws:

Exception

stop

public void stop()
          throws Exception

Description copied from interface: SubmittedJobGraphStore

Stops the SubmittedJobGraphStore service.

Specified by:

stop in interface SubmittedJobGraphStore

Throws:

Exception

recoverJobGraph

@Nullable
public SubmittedJobGraph recoverJobGraph(org.apache.flink.api.common.JobID jobId)
                                            throws Exception

Description copied from interface: SubmittedJobGraphStore

Returns the SubmittedJobGraph with the given JobID or null if no job was registered.

Specified by:

recoverJobGraph in interface SubmittedJobGraphStore

Throws:

Exception

putJobGraph

public void putJobGraph(SubmittedJobGraph jobGraph)
                 throws Exception

Description copied from interface: SubmittedJobGraphStore

Adds the SubmittedJobGraph instance.

If a job graph with the same JobID exists, it is replaced.

Specified by:

putJobGraph in interface SubmittedJobGraphStore

Throws:

Exception

removeJobGraph

public void removeJobGraph(org.apache.flink.api.common.JobID jobId)
                    throws Exception

Description copied from interface: SubmittedJobGraphStore

Removes the SubmittedJobGraph with the given JobID if it exists.

Specified by:

removeJobGraph in interface SubmittedJobGraphStore

Throws:

Exception

getJobIds

public Collection<org.apache.flink.api.common.JobID> getJobIds()
                                                        throws Exception

Description copied from interface: SubmittedJobGraphStore

Get all job ids of submitted job graphs to the submitted job graph store.

Specified by:

getJobIds in interface SubmittedJobGraphStore

Returns:

Collection of submitted job ids

Throws:

Exception - if the operation fails

getPathForJob

public static String getPathForJob(org.apache.flink.api.common.JobID jobId)

Returns the JobID as a String (with leading slash).

jobIdfromPath

public static org.apache.flink.api.common.JobID jobIdfromPath(String path)

Returns the JobID from the given path in ZooKeeper.

Parameters:

path - in ZooKeeper

Returns:

JobID associated with the given path