org.apache.gobblin.service.modules.flowgraph

Class BaseFlowGraph

java.lang.Object

org.apache.gobblin.service.modules.flowgraph.BaseFlowGraph

All Implemented Interfaces:

FlowGraph

@Alpha
public class BaseFlowGraph
extends Object
implements FlowGraph

A thread-safe implementation of FlowGraph. The implementation maintains the following data structures:

dataNodeMap - the mapping from a node identifier to the DataNode instance

nodesToEdges - the mapping from each DataNode to its outgoing FlowEdges

flowEdgeMap - the mapping from a edge label to the FlowEdge instance

Read/Write Access to the FlowGraph is synchronized via a ReentrantReadWriteLock.

Constructor Summary

Constructors

Constructor and Description
BaseFlowGraph()

Method Summary

Modifier and Type	Method and Description
boolean	addDataNode(DataNode node) Add a DataNode to the FlowGraph.
boolean	addFlowEdge(FlowEdge edge) Add a FlowEdge to the FlowGraph.
boolean	deleteDataNode(DataNode node) Delete a from the FlowGraph.
boolean	deleteDataNode(String nodeId) Delete a DataNode by its identifier
boolean	deleteFlowEdge(FlowEdge edge) Delete a from the FlowGraph.
boolean	deleteFlowEdge(String edgeId) Delete a DataNode by its identifier
FlowGraphPath	findPath(FlowSpec flowSpec) A method that takes a FlowSpec containing the source and destination DataNodes, as well as the source and target DatasetDescriptors, and returns a sequence of fully resolved JobSpecs that will move the source dataset from the source datanode, perform any necessary transformations and land the dataset at the destination node in the format described by the target DatasetDescriptor.
Set<FlowEdge>	getEdges(DataNode node) Get the set of edges adjacent to a DataNode
Set<FlowEdge>	getEdges(String nodeId) Get the set of edges adjacent to a DataNode
DataNode	getNode(String nodeId) Lookup a node by its identifier.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BaseFlowGraph

public BaseFlowGraph()

Method Detail

getNode

public DataNode getNode(String nodeId)

Lookup a node by its identifier.

Specified by:

getNode in interface FlowGraph

Parameters:

nodeId - node identifier

Returns:

DataNode with nodeId as the identifier.

addDataNode

public boolean addDataNode(DataNode node)

Add a DataNode to the FlowGraph. If the node already "exists" in the FlowGraph (i.e. the FlowGraph already has another node with the same id), we remove the old node and add the new one. The edges incident on the old node are preserved.

Specified by:

addDataNode in interface FlowGraph

Parameters:

node - to be added to the FlowGraph

Returns:

true if node is successfully added to the FlowGraph.

addFlowEdge

public boolean addFlowEdge(FlowEdge edge)

Add a FlowEdge to the FlowGraph. Addition of edge succeeds only if both the end points of the edge are already nodes in the FlowGraph. If a FlowEdge already exists, the old FlowEdge is removed and the new one added in its place.

Specified by:

addFlowEdge in interface FlowGraph

Parameters:

edge -

Returns:

true if addition of is successful.

deleteDataNode

public boolean deleteDataNode(String nodeId)

Delete a DataNode by its identifier

Specified by:

deleteDataNode in interface FlowGraph

Parameters:

nodeId - identifier of the DataNode to be deleted.

Returns:

true if DataNode is successfully deleted.

deleteDataNode

public boolean deleteDataNode(DataNode node)

Delete a from the FlowGraph.

Parameters:

node - to be deleted.

Returns:

true if DataNode is successfully deleted.

deleteFlowEdge

public boolean deleteFlowEdge(String edgeId)

Delete a DataNode by its identifier

Specified by:

deleteFlowEdge in interface FlowGraph

Parameters:

edgeId - identifier of the FlowEdge to be deleted.

Returns:

true if FlowEdge is successfully deleted.

deleteFlowEdge

public boolean deleteFlowEdge(FlowEdge edge)

Delete a from the FlowGraph.

Parameters:

edge - to be deleted.

Returns:

true if FlowEdge is successfully deleted. If the source of a FlowEdge does not exist or if the FlowEdge is not in the graph, return false.

getEdges

public Set<FlowEdge> getEdges(String nodeId)

Get the set of edges adjacent to a DataNode

Specified by:

getEdges in interface FlowGraph

Parameters:

nodeId - identifier of the node

Returns:

Set of FlowEdges adjacent to the node.

getEdges

public Set<FlowEdge> getEdges(DataNode node)

Get the set of edges adjacent to a DataNode

Specified by:

getEdges in interface FlowGraph

Parameters:

node - DataNode

Returns:

Set of FlowEdges adjacent to the node.

findPath

public FlowGraphPath findPath(FlowSpec flowSpec)
                       throws PathFinder.PathFinderException,
                              ReflectiveOperationException

A method that takes a FlowSpec containing the source and destination DataNodes, as well as the source and target DatasetDescriptors, and returns a sequence of fully resolved JobSpecs that will move the source dataset from the source datanode, perform any necessary transformations and land the dataset at the destination node in the format described by the target DatasetDescriptor.

Specified by:

findPath in interface FlowGraph

Parameters:

flowSpec - a Spec containing a high-level description of input flow.

Returns:

an instance of FlowGraphPath that encapsulates a sequence of JobSpecs satisfying flowSpec.

Throws:

PathFinder.PathFinderException

ReflectiveOperationException