程序包 org.apache.pulsar.io.hdfs3.sink.seq

类 HdfsAbstractSequenceFileSink<K,V,HdfsK,HdfsV>

java.lang.Object

org.apache.pulsar.io.hdfs3.AbstractHdfsConnector

org.apache.pulsar.io.hdfs3.sink.HdfsAbstractSink<K,V>

org.apache.pulsar.io.hdfs3.sink.seq.HdfsAbstractSequenceFileSink<K,V,HdfsK,HdfsV>

类型参数:

K - - The incoming Key type

V - - The incoming Value type

HdfsK - - The HDFS Key type

HdfsV - - The HDFS Value type

所有已实现的接口:

AutoCloseable, org.apache.pulsar.io.core.Sink<V>

直接已知子类:

HdfsSequentialTextSink, HdfsTextSink

public abstract class HdfsAbstractSequenceFileSink<K,V,HdfsK,HdfsV>
extends HdfsAbstractSink<K,V>
implements org.apache.pulsar.io.core.Sink<V>

HDFS Sink that writes it contents to HDFS as Sequence Files.

字段概要

字段

修饰符和类型	字段	说明
protected AtomicLong	counter
protected org.apache.hadoop.fs.FSDataOutputStream	hdfsStream
protected org.apache.hadoop.io.SequenceFile.Writer	writer

从类继承的字段 org.apache.pulsar.io.hdfs3.sink.HdfsAbstractSink

hdfsSinkConfig, syncThread, unackedRecords

从类继承的字段 org.apache.pulsar.io.hdfs3.AbstractHdfsConnector

compressionCodecFactory, connectorConfig, hdfsResources

构造器概要

构造器

构造器	说明
HdfsAbstractSequenceFileSink()

方法概要

修饰符和类型	方法	说明
void	close()
abstract org.apache.pulsar.io.core.KeyValue<HdfsK,HdfsV>	convert(org.apache.pulsar.io.core.KeyValue<K,V> kv)
protected void	createWriter()
protected List<org.apache.hadoop.io.SequenceFile.Writer.Option>	getOptions()
protected org.apache.hadoop.io.SequenceFile.Writer	getWriter()
void	write(org.apache.pulsar.functions.api.Record<V> record)

从类继承的方法 org.apache.pulsar.io.hdfs3.sink.HdfsAbstractSink

connectToHdfs, extractKeyValue, getHdfsStream, getOutputStreamBuilder, getPath, launchSyncThread, open

从类继承的方法 org.apache.pulsar.io.hdfs3.AbstractHdfsConnector

checkHdfsUriForTimeout, getCompressionCodec, getCompressionCodecFactory, getConfiguration, getEncoding, getFileSystem, getFileSystem, getFileSystemAsUser, getUserGroupInformation, resetHDFSResources

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

从接口继承的方法 org.apache.pulsar.io.core.Sink

open

字段详细资料

counter

protected AtomicLong counter

hdfsStream

protected org.apache.hadoop.fs.FSDataOutputStream hdfsStream

writer

protected org.apache.hadoop.io.SequenceFile.Writer writer

构造器详细资料

HdfsAbstractSequenceFileSink

public HdfsAbstractSequenceFileSink()

方法详细资料

convert

public abstract org.apache.pulsar.io.core.KeyValue<HdfsK,HdfsV> convert(org.apache.pulsar.io.core.KeyValue<K,V> kv)

close

public void close()
           throws Exception

指定者:

close 在接口中 AutoCloseable

覆盖:

close 在类中 HdfsAbstractSink<K,V>

抛出:

Exception

createWriter

protected void createWriter()
                     throws IOException

指定者:

createWriter 在类中 HdfsAbstractSink<K,V>

抛出:

IOException

write

public void write(org.apache.pulsar.functions.api.Record<V> record)

指定者:

write 在接口中 org.apache.pulsar.io.core.Sink<K>

getWriter

protected org.apache.hadoop.io.SequenceFile.Writer getWriter()
                                                      throws IOException

抛出:

IOException

getOptions

protected List<org.apache.hadoop.io.SequenceFile.Writer.Option> getOptions()
                                                                    throws IllegalArgumentException,
IOException

抛出:

IllegalArgumentException

IOException