org.apache.gobblin.service.modules.flowgraph.datanodes.fs

Class FileSystemDataNode

java.lang.Object

org.apache.gobblin.service.modules.flowgraph.BaseDataNode

org.apache.gobblin.service.modules.flowgraph.datanodes.fs.FileSystemDataNode

All Implemented Interfaces:

DataNode

Direct Known Subclasses:

AdlsDataNode, HdfsDataNode, LocalFSDataNode, SftpDataNode

@Alpha
public abstract class FileSystemDataNode
extends BaseDataNode

An abstract FileSystemDataNode implementation. In addition to the required properties of a BaseDataNode, an FileSystemDataNode must have a FS URI specified. Example implementations of FileSystemDataNode include HdfsDataNode, LocalFSDataNode.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.gobblin.service.modules.flowgraph.DataNode

DataNode.DataNodeCreationException

Field Summary

Fields

Modifier and Type	Field and Description
static String	FS_URI_KEY

Constructor Summary

Constructors

Constructor and Description
FileSystemDataNode(com.typesafe.config.Config nodeProps) Constructor.

Method Summary

Modifier and Type	Method and Description
String	getDefaultDatasetDescriptorClass()
abstract boolean	isUriValid(URI fsUri)

Methods inherited from class org.apache.gobblin.service.modules.flowgraph.BaseDataNode

getDefaultDatasetDescriptorPlatform

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.gobblin.service.modules.flowgraph.DataNode

getId, getRawConfig, isActive

Field Detail

FS_URI_KEY

public static final String FS_URI_KEY

See Also:

Constant Field Values

Constructor Detail

FileSystemDataNode

public FileSystemDataNode(com.typesafe.config.Config nodeProps)
                   throws DataNode.DataNodeCreationException

Constructor. An HDFS DataNode must have fs.uri property specified in addition to a node Id.

Throws:

DataNode.DataNodeCreationException

Method Detail

isUriValid

public abstract boolean isUriValid(URI fsUri)

getDefaultDatasetDescriptorClass

public String getDefaultDatasetDescriptorClass()

Specified by:

getDefaultDatasetDescriptorClass in interface DataNode

Overrides:

getDefaultDatasetDescriptorClass in class BaseDataNode

Returns:

a default dataset descriptor class for this DataNode, or null if a default should not be used.