org.apache.gobblin.service.modules.template

Interface FlowTemplate

All Superinterfaces:

Serializable, Spec

All Known Implementing Classes:

HOCONInputStreamFlowTemplate, StaticFlowTemplate

@Alpha
public interface FlowTemplate
extends Spec

An interface primarily for representing a flow of JobTemplates. It also has method for retrieving required configs for every JobTemplate in the flow.

Method Summary

Modifier and Type	Method and Description
List<org.apache.commons.lang3.tuple.Pair<DatasetDescriptor,DatasetDescriptor>>	getDatasetDescriptors(com.typesafe.config.Config userConfig, boolean resolvable)
List<JobTemplate>	getJobTemplates()
com.typesafe.config.Config	getRawTemplateConfig()
List<com.typesafe.config.Config>	getResolvedJobConfigs(com.typesafe.config.Config userConfig, DatasetDescriptor inputDescriptor, DatasetDescriptor outputDescriptor) Resolves the JobTemplates underlying this FlowTemplate and returns a List of resolved job Configs.
void	tryResolving(com.typesafe.config.Config userConfig, DatasetDescriptor inputDescriptor, DatasetDescriptor outputDescriptor) Try to resolve the FlowTemplate using the provided Config object.

Methods inherited from interface org.apache.gobblin.runtime.api.Spec

getDescription, getUri, getVersion

Method Detail

getJobTemplates

List<JobTemplate> getJobTemplates()

Returns:

the Collection of JobTemplates that belong to this FlowTemplate.

getRawTemplateConfig

com.typesafe.config.Config getRawTemplateConfig()

Returns:

all configuration inside pre-written template.

getDatasetDescriptors

List<org.apache.commons.lang3.tuple.Pair<DatasetDescriptor,DatasetDescriptor>> getDatasetDescriptors(com.typesafe.config.Config userConfig,
                                                                                                     boolean resolvable)
                                                                                              throws IOException,
                                                                                                     ReflectiveOperationException,
                                                                                                     SpecNotFoundException,
                                                                                                     JobTemplate.TemplateException

Parameters:

userConfig - a list of user customized attributes.

resolvable - if true, only return descriptors that resolve the FlowTemplate

Returns:

list of input/output DatasetDescriptors corresponding to the provied userConfig.

Throws:

IOException

ReflectiveOperationException

SpecNotFoundException

JobTemplate.TemplateException

tryResolving

void tryResolving(com.typesafe.config.Config userConfig,
                  DatasetDescriptor inputDescriptor,
                  DatasetDescriptor outputDescriptor)
           throws SpecNotFoundException,
                  JobTemplate.TemplateException

Try to resolve the FlowTemplate using the provided Config object. A FlowTemplate is resolvable only if each of the JobTemplates in the flow is resolvable. Throws an exception if the flow is not resolvable.

Parameters:

userConfig - User supplied Config

inputDescriptor - input DatasetDescriptor

outputDescriptor - output DatasetDescriptor

Throws:

SpecNotFoundException

JobTemplate.TemplateException

getResolvedJobConfigs

List<com.typesafe.config.Config> getResolvedJobConfigs(com.typesafe.config.Config userConfig,
                                                       DatasetDescriptor inputDescriptor,
                                                       DatasetDescriptor outputDescriptor)
                                                throws SpecNotFoundException,
                                                       JobTemplate.TemplateException

Resolves the JobTemplates underlying this FlowTemplate and returns a List of resolved job Configs.

Parameters:

userConfig - User supplied Config

inputDescriptor - input DatasetDescriptor

outputDescriptor - output DatasetDescriptor

Returns:

a list of resolved job Configs.

Throws:

SpecNotFoundException

JobTemplate.TemplateException