org.apache.gobblin.service.modules.flow

Class IdentityFlowToJobSpecCompiler

java.lang.Object

org.apache.gobblin.service.modules.flow.BaseFlowToJobSpecCompiler

org.apache.gobblin.service.modules.flow.IdentityFlowToJobSpecCompiler

All Implemented Interfaces:

Instrumentable, SpecCatalogListener, SpecCompiler

Direct Known Subclasses:

MockedSpecCompiler

@Alpha
public class IdentityFlowToJobSpecCompiler
extends BaseFlowToJobSpecCompiler

Take in a logical Spec ie flow and compile corresponding materialized job Spec and its mapping to SpecExecutor.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.gobblin.runtime.api.SpecCatalogListener

SpecCatalogListener.AddSpecCallback, SpecCatalogListener.DeleteSpecCallback, SpecCatalogListener.UpdateSpecCallback

Field Summary

Fields inherited from class org.apache.gobblin.service.modules.flow.BaseFlowToJobSpecCompiler

active, config, dataAuthorizationTimer, flowCompilationFailedMeter, flowCompilationSuccessFulMeter, flowCompilationTimer, log, metricContext, templateCatalog, topologySpecMap

Constructor Summary

Constructors

Constructor and Description
IdentityFlowToJobSpecCompiler(com.typesafe.config.Config config)
IdentityFlowToJobSpecCompiler(com.typesafe.config.Config config, boolean instrumentationEnabled)
IdentityFlowToJobSpecCompiler(com.typesafe.config.Config config, com.google.common.base.Optional<org.slf4j.Logger> log)
IdentityFlowToJobSpecCompiler(com.typesafe.config.Config config, com.google.common.base.Optional<org.slf4j.Logger> log, boolean instrumentationEnabled)

Method Summary

Modifier and Type	Method and Description
Dag<JobExecutionPlan>	compileFlow(Spec spec) Take in a logical Spec and compile corresponding materialized Specs and the mapping to SpecExecutor that they can be run on.

Methods inherited from class org.apache.gobblin.service.modules.flow.BaseFlowToJobSpecCompiler

awaitHealthy, generateTags, getMetricContext, getTopologySpecMap, isInstrumentationEnabled, jobSpecGenerator, jobSpecTemplateURIGenerator, jobSpecURIGenerator, onAddSpec, onDeleteSpec, onDeleteSpec, onUpdateSpec, switchMetricContext, switchMetricContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.gobblin.service.modules.flow.SpecCompiler

setActive

Methods inherited from interface org.apache.gobblin.runtime.api.SpecCatalogListener

getName

Constructor Detail

IdentityFlowToJobSpecCompiler

public IdentityFlowToJobSpecCompiler(com.typesafe.config.Config config)

IdentityFlowToJobSpecCompiler

public IdentityFlowToJobSpecCompiler(com.typesafe.config.Config config,
                                     boolean instrumentationEnabled)

IdentityFlowToJobSpecCompiler

public IdentityFlowToJobSpecCompiler(com.typesafe.config.Config config,
                                     com.google.common.base.Optional<org.slf4j.Logger> log)

IdentityFlowToJobSpecCompiler

public IdentityFlowToJobSpecCompiler(com.typesafe.config.Config config,
                                     com.google.common.base.Optional<org.slf4j.Logger> log,
                                     boolean instrumentationEnabled)

Method Detail

compileFlow

public Dag<JobExecutionPlan> compileFlow(Spec spec)

Description copied from interface: SpecCompiler

Take in a logical Spec and compile corresponding materialized Specs and the mapping to SpecExecutor that they can be run on. All the specs generated from the compileFlow must have a "flow.executionId"

Specified by:

compileFlow in interface SpecCompiler

Specified by:

compileFlow in class BaseFlowToJobSpecCompiler

Parameters:

spec - Spec to compile.

Returns:

Map of materialized physical Spec and SpecExecutor.