org.apache.gobblin.service.modules.flowgraph.datanodes.fs

Class HdfsDataNode

java.lang.Object

org.apache.gobblin.service.modules.flowgraph.BaseDataNode

org.apache.gobblin.service.modules.flowgraph.datanodes.fs.FileSystemDataNode

org.apache.gobblin.service.modules.flowgraph.datanodes.fs.HdfsDataNode

All Implemented Interfaces:

DataNode

@Alpha
public class HdfsDataNode
extends FileSystemDataNode

An implementation of HdfsDataNode. All the properties specific to a HDFS based data node (e.g. fs.uri) are validated here.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.gobblin.service.modules.flowgraph.DataNode

DataNode.DataNodeCreationException

Field Summary

Fields

Modifier and Type	Field and Description
static String	HDFS_SCHEME

Fields inherited from class org.apache.gobblin.service.modules.flowgraph.datanodes.fs.FileSystemDataNode

FS_URI_KEY

Constructor Summary

Constructors

Constructor and Description
HdfsDataNode(com.typesafe.config.Config nodeProps)

Method Summary

Modifier and Type	Method and Description
String	getDefaultDatasetDescriptorPlatform()
boolean	isUriValid(URI fsUri)

Methods inherited from class org.apache.gobblin.service.modules.flowgraph.datanodes.fs.FileSystemDataNode

getDefaultDatasetDescriptorClass

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.gobblin.service.modules.flowgraph.DataNode

getId, getRawConfig, isActive

Field Detail

HDFS_SCHEME

public static final String HDFS_SCHEME

See Also:

Constant Field Values

Constructor Detail

HdfsDataNode

public HdfsDataNode(com.typesafe.config.Config nodeProps)
             throws DataNode.DataNodeCreationException

Throws:

DataNode.DataNodeCreationException

Method Detail

isUriValid

public boolean isUriValid(URI fsUri)

Specified by:

isUriValid in class FileSystemDataNode

Parameters:

fsUri - FileSystem URI

Returns:

true if the scheme is "hdfs" and authority is not empty.

getDefaultDatasetDescriptorPlatform

public String getDefaultDatasetDescriptorPlatform()

Specified by:

getDefaultDatasetDescriptorPlatform in interface DataNode

Overrides:

getDefaultDatasetDescriptorPlatform in class BaseDataNode

Returns:

a default dataset descriptor platform for this DataNode, or null if a default should not be used.