@Alpha public class GobblinServiceJobScheduler extends JobScheduler implements SpecCatalogListener
JobScheduler that is also a SpecCatalogListener.
GobblinServiceJobScheduler listens for new / updated FlowSpec and schedules
and runs them via Orchestrator.| Modifier and Type | Class and Description |
|---|---|
static class |
GobblinServiceJobScheduler.GobblinServiceJob
A Gobblin job to be scheduled.
|
JobScheduler.Action, JobScheduler.GobblinJobSpecCatalogListener.AddSpecCallback, SpecCatalogListener.DeleteSpecCallback, SpecCatalogListener.UpdateSpecCallback| Modifier and Type | Field and Description |
|---|---|
protected org.slf4j.Logger |
_log |
static String |
DR_FILTER_TAG
Use this to tag all DR-applicable FlowSpec entries in
SpecStore
so only they would be loaded during DR handling. |
protected com.google.common.base.Optional<FlowCatalog> |
flowCatalog |
static String |
GOBBLIN_SERVICE_SCHEDULER_DR_NOMINATED |
protected com.google.common.base.Optional<org.apache.helix.HelixManager> |
helixManager |
protected Orchestrator |
orchestrator |
protected Map<String,Spec> |
scheduledFlowSpecs |
JOB_LISTENER_KEY, JOB_SCHEDULER_KEY, jobConfigFileDirPath, jobConfigFileExtensions, jobExecutor, listener, pathAlterationDetector, properties, PROPERTIES_KEY| Constructor and Description |
|---|
GobblinServiceJobScheduler(String serviceName,
com.typesafe.config.Config config,
com.google.common.base.Optional<org.apache.helix.HelixManager> helixManager,
com.google.common.base.Optional<FlowCatalog> flowCatalog,
com.google.common.base.Optional<TopologyCatalog> topologyCatalog,
com.google.common.base.Optional<DagManager> dagManager,
SchedulerService schedulerService,
com.google.common.base.Optional<org.slf4j.Logger> log) |
GobblinServiceJobScheduler(String serviceName,
com.typesafe.config.Config config,
com.google.common.base.Optional<org.apache.helix.HelixManager> helixManager,
com.google.common.base.Optional<FlowCatalog> flowCatalog,
com.google.common.base.Optional<TopologyCatalog> topologyCatalog,
Orchestrator orchestrator,
SchedulerService schedulerService,
com.google.common.base.Optional<org.slf4j.Logger> log) |
| Modifier and Type | Method and Description |
|---|---|
protected static Spec |
disableFlowRunImmediatelyOnStart(FlowSpec spec) |
AddSpecResponse |
onAddSpec(Spec addedSpec) |
void |
onDeleteSpec(URI deletedSpecURI,
String deletedSpecVersion) |
void |
onDeleteSpec(URI deletedSpecURI,
String deletedSpecVersion,
Properties headers) |
void |
onUpdateSpec(Spec updatedSpec) |
void |
runJob(Properties jobProps,
JobListener jobListener) |
void |
scheduleJob(Properties jobProps,
JobListener jobListener)
Synchronize the job scheduling because the same flowSpec can be scheduled by different threads.
|
void |
setActive(boolean isActive) |
protected void |
startUp() |
buildJobLauncher, getJobSpecResolver, getScheduledJobs, isCancelRequested, runJob, scheduleJob, scheduleJobImmediately, scheduleJobImmediately, shutDown, startServices, submitRunnableToExecutor, unscheduleJobaddListener, awaitRunning, awaitRunning, awaitTerminated, awaitTerminated, executor, failureCause, isRunning, serviceName, start, startAndWait, startAsync, state, stop, stopAndWait, stopAsync, toStringclone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, waitgetNamepublic static final String GOBBLIN_SERVICE_SCHEDULER_DR_NOMINATED
protected final org.slf4j.Logger _log
protected final com.google.common.base.Optional<FlowCatalog> flowCatalog
protected final com.google.common.base.Optional<org.apache.helix.HelixManager> helixManager
protected final Orchestrator orchestrator
public static final String DR_FILTER_TAG
SpecStore
so only they would be loaded during DR handling.public GobblinServiceJobScheduler(String serviceName, com.typesafe.config.Config config, com.google.common.base.Optional<org.apache.helix.HelixManager> helixManager, com.google.common.base.Optional<FlowCatalog> flowCatalog, com.google.common.base.Optional<TopologyCatalog> topologyCatalog, Orchestrator orchestrator, SchedulerService schedulerService, com.google.common.base.Optional<org.slf4j.Logger> log) throws Exception
Exceptionpublic GobblinServiceJobScheduler(String serviceName, com.typesafe.config.Config config, com.google.common.base.Optional<org.apache.helix.HelixManager> helixManager, com.google.common.base.Optional<FlowCatalog> flowCatalog, com.google.common.base.Optional<TopologyCatalog> topologyCatalog, com.google.common.base.Optional<DagManager> dagManager, SchedulerService schedulerService, com.google.common.base.Optional<org.slf4j.Logger> log) throws Exception
Exceptionpublic void setActive(boolean isActive)
protected static Spec disableFlowRunImmediatelyOnStart(FlowSpec spec)
protected void startUp()
throws Exception
startUp in class JobSchedulerExceptionpublic void scheduleJob(Properties jobProps, JobListener jobListener) throws JobException
scheduleJob in class JobSchedulerJobExceptionpublic void runJob(Properties jobProps, JobListener jobListener) throws JobException
runJob in class JobSchedulerJobExceptionpublic AddSpecResponse onAddSpec(Spec addedSpec)
onAddSpec in interface SpecCatalogListeneraddedSpec - spec to be addedpublic void onDeleteSpec(URI deletedSpecURI, String deletedSpecVersion, Properties headers)
onDeleteSpec in interface SpecCatalogListenerpublic void onUpdateSpec(Spec updatedSpec)
onUpdateSpec in interface SpecCatalogListener