程序包 org.apache.pulsar.io.hdfs3.sink.seq

类 HdfsSequentialTextSink

java.lang.Object

org.apache.pulsar.io.hdfs3.AbstractHdfsConnector

org.apache.pulsar.io.hdfs3.sink.HdfsAbstractSink<K,V>

org.apache.pulsar.io.hdfs3.sink.seq.HdfsAbstractSequenceFileSink<Long,String,org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text>

org.apache.pulsar.io.hdfs3.sink.seq.HdfsSequentialTextSink

所有已实现的接口:

AutoCloseable, org.apache.pulsar.io.core.Sink<String>

public class HdfsSequentialTextSink
extends HdfsAbstractSequenceFileSink<Long,String,org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text>

This Sink should be used when the records are originating from a sequential source, and we want to retain the record sequence.This class uses the record's sequence id as the sequence id in the HDFS Sequence File if it is available, if not a sequence id is auto-generated for each new record.

字段概要

从类继承的字段 org.apache.pulsar.io.hdfs3.sink.seq.HdfsAbstractSequenceFileSink

hdfsStream, writer

从类继承的字段 org.apache.pulsar.io.hdfs3.sink.HdfsAbstractSink

hdfsSinkConfig, syncThread, unackedRecords

从类继承的字段 org.apache.pulsar.io.hdfs3.AbstractHdfsConnector

compressionCodecFactory, connectorConfig, hdfsResources

构造器概要

构造器

构造器	说明
HdfsSequentialTextSink()

方法概要

修饰符和类型	方法	说明
org.apache.pulsar.io.core.KeyValue<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text>	convert(org.apache.pulsar.io.core.KeyValue<Long,String> kv)
org.apache.pulsar.io.core.KeyValue<Long,String>	extractKeyValue(org.apache.pulsar.functions.api.Record<String> record)
protected List<org.apache.hadoop.io.SequenceFile.Writer.Option>	getOptions()
org.apache.hadoop.io.SequenceFile.Writer	getWriter()

从类继承的方法 org.apache.pulsar.io.hdfs3.sink.seq.HdfsAbstractSequenceFileSink

close, createWriter, write

从类继承的方法 org.apache.pulsar.io.hdfs3.sink.HdfsAbstractSink

connectToHdfs, getHdfsStream, getOutputStreamBuilder, getPath, launchSyncThread, open

从类继承的方法 org.apache.pulsar.io.hdfs3.AbstractHdfsConnector

checkHdfsUriForTimeout, getCompressionCodec, getCompressionCodecFactory, getConfiguration, getEncoding, getFileSystem, getFileSystem, getFileSystemAsUser, getUserGroupInformation, resetHDFSResources

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

从接口继承的方法 org.apache.pulsar.io.core.Sink

open

构造器详细资料

HdfsSequentialTextSink

public HdfsSequentialTextSink()

方法详细资料

getWriter

public org.apache.hadoop.io.SequenceFile.Writer getWriter()
                                                   throws IOException

覆盖:

getWriter 在类中 HdfsAbstractSequenceFileSink<Long,String,org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text>

抛出:

IOException

getOptions

protected List<org.apache.hadoop.io.SequenceFile.Writer.Option> getOptions()
                                                                    throws IllegalArgumentException,
IOException

覆盖:

getOptions 在类中 HdfsAbstractSequenceFileSink<Long,String,org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text>

抛出:

IllegalArgumentException

IOException

extractKeyValue

public org.apache.pulsar.io.core.KeyValue<Long,String> extractKeyValue(org.apache.pulsar.functions.api.Record<String> record)

指定者:

extractKeyValue 在类中 HdfsAbstractSink<Long,String>

convert

public org.apache.pulsar.io.core.KeyValue<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text> convert(org.apache.pulsar.io.core.KeyValue<Long,String> kv)

指定者:

convert 在类中 HdfsAbstractSequenceFileSink<Long,String,org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text>