程序包 org.apache.pulsar.io.hdfs3

类 AbstractHdfsConnector

java.lang.Object
org.apache.pulsar.io.hdfs3.AbstractHdfsConnector
直接已知子类:
HdfsAbstractSink
public abstract class AbstractHdfsConnector extends Object
A Simple abstract class for HDFS connectors. Provides methods for connecting to HDFS

  • 字段详细资料

    • hdfsResources

      protected final AtomicReference<HdfsResources> hdfsResources

    • connectorConfig

      protected AbstractHdfsConfig connectorConfig

    • compressionCodecFactory

      protected org.apache.hadoop.io.compress.CompressionCodecFactory compressionCodecFactory

  • 构造器详细资料

    • AbstractHdfsConnector

      public AbstractHdfsConnector()

  • 方法详细资料

    • resetHDFSResources

      protected HdfsResources resetHDFSResources(HdfsSinkConfig hdfsSinkConfig) throws IOException
      抛出:
      IOException

    • checkHdfsUriForTimeout

      protected void checkHdfsUriForTimeout(org.apache.hadoop.conf.Configuration config) throws IOException
      抛出:
      IOException

    • getFileSystem

      protected org.apache.hadoop.fs.FileSystem getFileSystem(org.apache.hadoop.conf.Configuration config) throws IOException
      This exists in order to allow unit tests to override it so that they don't take several minutes waiting for UDP packets to be received.
      参数:
      config - the configuration to use
      返回:
      the FileSystem that is created for the given Configuration
      抛出:
      IOException - if unable to create the FileSystem

    • getFileSystemAsUser

      protected org.apache.hadoop.fs.FileSystem getFileSystemAsUser(org.apache.hadoop.conf.Configuration config, org.apache.hadoop.security.UserGroupInformation ugi) throws IOException
      抛出:
      IOException

    • getConfiguration

      protected org.apache.hadoop.conf.Configuration getConfiguration()

    • getFileSystem

      protected org.apache.hadoop.fs.FileSystem getFileSystem()

    • getUserGroupInformation

      protected org.apache.hadoop.security.UserGroupInformation getUserGroupInformation()

    • getEncoding

      protected String getEncoding()

    • getCompressionCodec

      protected org.apache.hadoop.io.compress.CompressionCodec getCompressionCodec()

    • getCompressionCodecFactory

      protected org.apache.hadoop.io.compress.CompressionCodecFactory getCompressionCodecFactory()