org.apache.flink.streaming.connectors.fs

Class StreamWriterBase<T>

java.lang.Object

org.apache.flink.streaming.connectors.fs.StreamWriterBase<T>

All Implemented Interfaces:

Serializable, Writer<T>

Direct Known Subclasses:

AvroKeyValueSinkWriter, SequenceFileWriter, StringWriter

public abstract class StreamWriterBase<T>
extends Object
implements Writer<T>

Base class for Writers that write to a FSDataOutputStream.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
StreamWriterBase()

Method Summary

Modifier and Type	Method and Description
void	close() Closes the Writer.
long	flush() Flushes out any internally held data, and returns the offset that the file must be truncated to at recovery.
long	getPos() Retrieves the current position, and thus size, of the output file.
protected org.apache.hadoop.fs.FSDataOutputStream	getStream() Returns the current output stream, if the stream is open.
protected void	hflushOrSync(org.apache.hadoop.fs.FSDataOutputStream os) If hflush is available in this version of HDFS, then this method calls hflush, else it calls sync.
void	open(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path) Initializes the Writer for a newly opened bucket file.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.streaming.connectors.fs.Writer

duplicate, write

Constructor Detail

StreamWriterBase

public StreamWriterBase()

Method Detail

getStream

protected org.apache.hadoop.fs.FSDataOutputStream getStream()

Returns the current output stream, if the stream is open.

hflushOrSync

protected void hflushOrSync(org.apache.hadoop.fs.FSDataOutputStream os)
                     throws IOException

If hflush is available in this version of HDFS, then this method calls hflush, else it calls sync.

Parameters:

os - - The stream to flush/sync

Throws:

IOException -

Note: This code comes from Flume

open

public void open(org.apache.hadoop.fs.FileSystem fs,
                 org.apache.hadoop.fs.Path path)
          throws IOException

Description copied from interface: Writer

Initializes the Writer for a newly opened bucket file. Any internal per-bucket initialization should be performed here.

Specified by:

open in interface Writer<T>

Parameters:

fs - The FileSystem containing the newly opened file.

path - The Path of the newly opened file.

Throws:

IOException

flush

public long flush()
           throws IOException

Description copied from interface: Writer

Flushes out any internally held data, and returns the offset that the file must be truncated to at recovery.

Specified by:

flush in interface Writer<T>

Throws:

IOException

getPos

public long getPos()
            throws IOException

Description copied from interface: Writer

Retrieves the current position, and thus size, of the output file.

Specified by:

getPos in interface Writer<T>

Throws:

IOException

close

public void close()
           throws IOException

Description copied from interface: Writer

Closes the Writer. If the writer is already closed, no action will be taken. The call should close all state related to the current output file, including the output stream opened in open.

Specified by:

close in interface Writer<T>

Throws:

IOException