org.apache.gobblin.service.modules.template

Class StaticFlowTemplate

java.lang.Object

org.apache.gobblin.service.modules.template.StaticFlowTemplate

All Implemented Interfaces:

Serializable, Spec, FlowTemplate

Direct Known Subclasses:

HOCONInputStreamFlowTemplate

@Alpha
public class StaticFlowTemplate
extends Object
implements FlowTemplate

A FlowTemplate using a static Config as the raw configuration for the template.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
StaticFlowTemplate(URI flowTemplateDirUri, String version, String description, com.typesafe.config.Config config, FlowCatalogWithTemplates catalog)
StaticFlowTemplate(URI uri, String version, String description, com.typesafe.config.Config config, FlowCatalogWithTemplates catalog, List<JobTemplate> jobTemplates)

Method Summary

Modifier and Type	Method and Description
List<org.apache.commons.lang3.tuple.Pair<DatasetDescriptor,DatasetDescriptor>>	getDatasetDescriptors(com.typesafe.config.Config userConfig, boolean resolvable) Generate the input/output dataset descriptors for the FlowTemplate.
List<JobTemplate>	getJobTemplates()
com.typesafe.config.Config	getRawTemplateConfig()
List<com.typesafe.config.Config>	getResolvedJobConfigs(com.typesafe.config.Config userConfig, DatasetDescriptor inputDescriptor, DatasetDescriptor outputDescriptor) Resolves the JobTemplates underlying this FlowTemplate and returns a List of resolved job Configs.
void	tryResolving(com.typesafe.config.Config userConfig, DatasetDescriptor inputDescriptor, DatasetDescriptor outputDescriptor) Try to resolve the FlowTemplate using the provided Config object.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.gobblin.runtime.api.Spec

getDescription, getUri, getVersion

Constructor Detail

StaticFlowTemplate

public StaticFlowTemplate(URI flowTemplateDirUri,
                          String version,
                          String description,
                          com.typesafe.config.Config config,
                          FlowCatalogWithTemplates catalog)
                   throws IOException,
                          SpecNotFoundException,
                          JobTemplate.TemplateException,
                          URISyntaxException

Throws:

IOException

SpecNotFoundException

JobTemplate.TemplateException

URISyntaxException

StaticFlowTemplate

public StaticFlowTemplate(URI uri,
                          String version,
                          String description,
                          com.typesafe.config.Config config,
                          FlowCatalogWithTemplates catalog,
                          List<JobTemplate> jobTemplates)

Method Detail

getDatasetDescriptors

public List<org.apache.commons.lang3.tuple.Pair<DatasetDescriptor,DatasetDescriptor>> getDatasetDescriptors(com.typesafe.config.Config userConfig,
                                                                                                            boolean resolvable)
                                                                                                     throws IOException

Generate the input/output dataset descriptors for the FlowTemplate.

Specified by:

getDatasetDescriptors in interface FlowTemplate

Parameters:

userConfig - User supplied Config

resolvable - Whether to return only resolvable dataset descriptors

Returns:

a List of Input/Output DatasetDescriptors that correspond to this FlowTemplate. If resolvable is true, only return descriptors that fully resolve it.

Throws:

IOException

getRawTemplateConfig

public com.typesafe.config.Config getRawTemplateConfig()

Specified by:

getRawTemplateConfig in interface FlowTemplate

Returns:

all configuration inside pre-written template.

getJobTemplates

public List<JobTemplate> getJobTemplates()

Specified by:

getJobTemplates in interface FlowTemplate

Returns:

the Collection of JobTemplates that belong to this FlowTemplate.

tryResolving

public void tryResolving(com.typesafe.config.Config userConfig,
                         DatasetDescriptor inputDescriptor,
                         DatasetDescriptor outputDescriptor)
                  throws SpecNotFoundException,
                         JobTemplate.TemplateException

Try to resolve the FlowTemplate using the provided Config object. A FlowTemplate is resolvable only if each of the JobTemplates in the flow is resolvable. Throws an exception if the flow is not resolvable.

Specified by:

tryResolving in interface FlowTemplate

Parameters:

userConfig - User supplied Config

inputDescriptor - input DatasetDescriptor

outputDescriptor - output DatasetDescriptor

Throws:

SpecNotFoundException

JobTemplate.TemplateException

getResolvedJobConfigs

public List<com.typesafe.config.Config> getResolvedJobConfigs(com.typesafe.config.Config userConfig,
                                                              DatasetDescriptor inputDescriptor,
                                                              DatasetDescriptor outputDescriptor)
                                                       throws SpecNotFoundException,
                                                              JobTemplate.TemplateException

Description copied from interface: FlowTemplate

Resolves the JobTemplates underlying this FlowTemplate and returns a List of resolved job Configs.

Specified by:

getResolvedJobConfigs in interface FlowTemplate

Parameters:

userConfig - User supplied Config

inputDescriptor - input DatasetDescriptor

outputDescriptor - output DatasetDescriptor

Returns:

a list of resolved job Configs.

Throws:

SpecNotFoundException

JobTemplate.TemplateException