org.apache.gobblin.service.modules.flowgraph

Interface FlowGraph

All Known Implementing Classes:

BaseFlowGraph

@Alpha
public interface FlowGraph

An interface for FlowGraph. A FlowGraph consists of DataNodes and FlowEdges. The interface provides methods for adding and removing DataNodes and FlowEdges to the FlowGraph. In addition the interface provides methods to return factory classes for creation of DataNodes and FlowEdges.

Method Summary

Modifier and Type	Method and Description
boolean	addDataNode(DataNode node) Add a DataNode to the FlowGraph
boolean	addFlowEdge(FlowEdge edge) Add a FlowEdge to the FlowGraph
boolean	deleteDataNode(String nodeId) Remove a DataNode and all its incident edges from the FlowGraph
boolean	deleteFlowEdge(String edgeId) Remove a FlowEdge from the FlowGraph
FlowGraphPath	findPath(FlowSpec flowSpec) A method that takes a FlowSpec containing the source and destination DataNodes, as well as the source and target DatasetDescriptors, and returns a sequence of fully resolved JobSpecs that will move the source dataset from the source datanode, perform any necessary transformations and land the dataset at the destination node in the format described by the target DatasetDescriptor.
Collection<FlowEdge>	getEdges(DataNode node) Get a collection of edges adjacent to a DataNode.
Collection<FlowEdge>	getEdges(String nodeId) Get a collection of edges adjacent to a DataNode.
DataNode	getNode(String nodeId) Get a DataNode from the node identifier

Method Detail

getNode

DataNode getNode(String nodeId)

Get a DataNode from the node identifier

Parameters:

nodeId - DataNode identifier.

Returns:

the DataNode object if the node is present in the FlowGraph.

addDataNode

boolean addDataNode(DataNode node)

Add a DataNode to the FlowGraph

Parameters:

node - DataNode to be added

Returns:

true if DataNode is added to the FlowGraph successfully.

addFlowEdge

boolean addFlowEdge(FlowEdge edge)

Add a FlowEdge to the FlowGraph

Parameters:

edge - FlowEdge to be added

Returns:

true if FlowEdge is added to the FlowGraph successfully.

deleteDataNode

boolean deleteDataNode(String nodeId)

Remove a DataNode and all its incident edges from the FlowGraph

Parameters:

nodeId - identifier of the DataNode to be removed

Returns:

true if DataNode is removed from the FlowGraph successfully.

deleteFlowEdge

boolean deleteFlowEdge(String edgeId)

Remove a FlowEdge from the FlowGraph

Parameters:

edgeId - label of the edge to be removed

Returns:

true if edge is removed from the FlowGraph successfully.

getEdges

Collection<FlowEdge> getEdges(String nodeId)

Get a collection of edges adjacent to a DataNode. Useful for path finding algorithms and graph traversal algorithms such as Djikstra's shortest-path algorithm, BFS

Parameters:

nodeId - identifier of the DataNode

Returns:

a collection of edges adjacent to the DataNode

getEdges

Collection<FlowEdge> getEdges(DataNode node)

Get a collection of edges adjacent to a DataNode.

Parameters:

node - DataNode

Returns:

a collection of edges adjacent to the DataNode

findPath

FlowGraphPath findPath(FlowSpec flowSpec)
                throws PathFinder.PathFinderException,
                       ReflectiveOperationException

A method that takes a FlowSpec containing the source and destination DataNodes, as well as the source and target DatasetDescriptors, and returns a sequence of fully resolved JobSpecs that will move the source dataset from the source datanode, perform any necessary transformations and land the dataset at the destination node in the format described by the target DatasetDescriptor.

Parameters:

flowSpec - a Spec containing a high-level description of input flow.

Returns:

an instance of FlowGraphPath that encapsulates a sequence of JobSpecs satisfying flowSpec.

Throws:

PathFinder.PathFinderException

ReflectiveOperationException