co.cask.cdap.logging.write

Class AvroFileWriter

java.lang.Object

co.cask.cdap.logging.write.AvroFileWriter

All Implemented Interfaces:

Closeable, Flushable, AutoCloseable

public final class AvroFileWriter
extends Object
implements Closeable, Flushable

Helper class that manages writing of KafkaLogEvent to Avro files. The events are written into appropriate files based on the LoggingContext of the event. The files are also rotated based on size. This class is not thread-safe.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
class	AvroFileWriter.AvroFile Represents an Avro file.

Constructor Summary

Constructors

Constructor and Description
AvroFileWriter(FileMetaDataManager fileMetaDataManager, CConfiguration cConf, Location rootDir, String logBaseDir, Schema schema, long maxFileSize, int syncIntervalBytes, long inactiveIntervalMs) Constructs an AvroFileWriter object.

Method Summary

Methods

Modifier and Type	Method and Description
void	append(List<? extends LogWriteEvent> events) Appends a log event to an appropriate Avro file based on LoggingContext.
void	close()
void	flush()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

AvroFileWriter

public AvroFileWriter(FileMetaDataManager fileMetaDataManager,
              CConfiguration cConf,
              Location rootDir,
              String logBaseDir,
              Schema schema,
              long maxFileSize,
              int syncIntervalBytes,
              long inactiveIntervalMs)

Constructs an AvroFileWriter object.

Parameters:

fileMetaDataManager - used to store file meta data.

cConf - the CDAP configuration

rootDir - the CDAP root dir on the filesystem

logBaseDir - the basedirectory for logs as defined in configuration

schema - schema of the Avro data to be written.

maxFileSize - Avro files greater than maxFileSize will get rotated.

syncIntervalBytes - the approximate number of uncompressed bytes to write in each block.

inactiveIntervalMs - files that have no data written for more than inactiveIntervalMs will be closed.

Method Detail

append

public void append(List<? extends LogWriteEvent> events)
            throws Exception

Appends a log event to an appropriate Avro file based on LoggingContext. If the log event does not contain LoggingContext then the event will be dropped.

Parameters:

events - Log event

Throws:

IOException

Exception

close

public void close()
           throws IOException

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Throws:

IOException

flush

public void flush()
           throws IOException

Specified by:

flush in interface Flushable

Throws:

IOException