@Alpha public interface DagStateStore
Dags. In case of a leadership
change in the GobblinServiceManager, the corresponding DagManager
loads the running Dags from the DagStateStore to resume their execution.| Modifier and Type | Method and Description |
|---|---|
void |
cleanUp(Dag<JobExecutionPlan> dag)
Delete the
Dag from the backing store, typically upon completion of execution. |
List<Dag<JobExecutionPlan>> |
getDags()
Load all currently running
Dags from the underlying store. |
void |
writeCheckpoint(Dag<JobExecutionPlan> dag)
Persist the
Dag to the backing store. |
void writeCheckpoint(Dag<JobExecutionPlan> dag) throws IOException
Dag to the backing store.
This is not an actual checkpoint but more like a Write-ahead log, where uncommitted job will be persisted
and be picked up again when leader transition happens.dag - The dag submitted to DagManagerIOExceptionvoid cleanUp(Dag<JobExecutionPlan> dag) throws IOException
Dag from the backing store, typically upon completion of execution.dag - The dag completed/cancelled from execution on SpecExecutor.IOExceptionList<Dag<JobExecutionPlan>> getDags() throws IOException
Dags from the underlying store. Typically, invoked when a new DagManager
takes over or on restart of service.List of currently running Dags.IOException