程序包 org.apache.pulsar.io.hdfs3

类 AbstractHdfsConnector

java.lang.Object

org.apache.pulsar.io.hdfs3.AbstractHdfsConnector

直接已知子类:

HdfsAbstractSink

public abstract class AbstractHdfsConnector
extends java.lang.Object

A Simple abstract class for HDFS connectors. Provides methods for connecting to HDFS

字段概要

字段

修饰符和类型	字段	说明
protected org.apache.hadoop.io.compress.CompressionCodecFactory	compressionCodecFactory
protected AbstractHdfsConfig	connectorConfig
protected java.util.concurrent.atomic.AtomicReference<HdfsResources>	hdfsResources

构造器概要

构造器

构造器	说明
AbstractHdfsConnector()

方法概要

修饰符和类型	方法	说明
protected void	checkHdfsUriForTimeout(org.apache.hadoop.conf.Configuration config)
protected org.apache.hadoop.io.compress.CompressionCodec	getCompressionCodec()
protected org.apache.hadoop.io.compress.CompressionCodecFactory	getCompressionCodecFactory()
protected org.apache.hadoop.conf.Configuration	getConfiguration()
protected java.lang.String	getEncoding()
protected org.apache.hadoop.fs.FileSystem	getFileSystem()
protected org.apache.hadoop.fs.FileSystem	getFileSystem(org.apache.hadoop.conf.Configuration config)	This exists in order to allow unit tests to override it so that they don't take several minutes waiting for UDP packets to be received.
protected org.apache.hadoop.fs.FileSystem	getFileSystemAsUser(org.apache.hadoop.conf.Configuration config, org.apache.hadoop.security.UserGroupInformation ugi)
protected org.apache.hadoop.security.UserGroupInformation	getUserGroupInformation()
protected HdfsResources	resetHDFSResources(HdfsSinkConfig hdfsSinkConfig)

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

字段详细资料

hdfsResources

protected final java.util.concurrent.atomic.AtomicReference<HdfsResources> hdfsResources

connectorConfig

protected AbstractHdfsConfig connectorConfig

compressionCodecFactory

protected org.apache.hadoop.io.compress.CompressionCodecFactory compressionCodecFactory

构造器详细资料

AbstractHdfsConnector

public AbstractHdfsConnector()

方法详细资料

resetHDFSResources

protected HdfsResources resetHDFSResources(HdfsSinkConfig hdfsSinkConfig)
                                    throws java.io.IOException

抛出:

java.io.IOException

checkHdfsUriForTimeout

protected void checkHdfsUriForTimeout(org.apache.hadoop.conf.Configuration config)
                               throws java.io.IOException

抛出:

java.io.IOException

getFileSystem

protected org.apache.hadoop.fs.FileSystem getFileSystem(org.apache.hadoop.conf.Configuration config)
                                                 throws java.io.IOException

This exists in order to allow unit tests to override it so that they don't take several minutes waiting for UDP packets to be received.

参数:

config - the configuration to use

返回:

the FileSystem that is created for the given Configuration

抛出:

java.io.IOException - if unable to create the FileSystem

getFileSystemAsUser

protected org.apache.hadoop.fs.FileSystem getFileSystemAsUser(org.apache.hadoop.conf.Configuration config,
                                                              org.apache.hadoop.security.UserGroupInformation ugi)
                                                       throws java.io.IOException

抛出:

java.io.IOException

getConfiguration

protected org.apache.hadoop.conf.Configuration getConfiguration()

getFileSystem

protected org.apache.hadoop.fs.FileSystem getFileSystem()

getUserGroupInformation

protected org.apache.hadoop.security.UserGroupInformation getUserGroupInformation()

getEncoding

protected java.lang.String getEncoding()

getCompressionCodec

protected org.apache.hadoop.io.compress.CompressionCodec getCompressionCodec()

getCompressionCodecFactory

protected org.apache.hadoop.io.compress.CompressionCodecFactory getCompressionCodecFactory()