co.cask.cdap.logging.save

Class KafkaLogWriterPlugin

java.lang.Object

co.cask.cdap.logging.save.AbstractKafkaLogProcessor

co.cask.cdap.logging.save.KafkaLogWriterPlugin

All Implemented Interfaces:

KafkaLogProcessor

public class KafkaLogWriterPlugin
extends AbstractKafkaLogProcessor

Plugin that writes the log data.

Field Summary

Fields

Modifier and Type	Field and Description
static int	CHECKPOINT_ROW_KEY_PREFIX

Constructor Summary

Constructors

Constructor and Description
KafkaLogWriterPlugin(CConfiguration cConfig, FileMetaDataManager fileMetaDataManager, LocationFactory locationFactory, CheckpointManagerFactory checkpointManagerFactory)

Method Summary

Methods

Modifier and Type	Method and Description
void	doProcess(KafkaLogEvent event) doProcess method will be called if the event is not already processed.
Checkpoint	getCheckpoint(int partition) Get the checkpoint offset for a given partition.
void	init(Set<Integer> partitions) Called when the leader partitions change.
void	stop() Called to stop processing for the current set of kafka partitions.

Methods inherited from class co.cask.cdap.logging.save.AbstractKafkaLogProcessor

alreadyProcessed, init, process

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

CHECKPOINT_ROW_KEY_PREFIX

public static final int CHECKPOINT_ROW_KEY_PREFIX

See Also:

Constant Field Values

Constructor Detail

KafkaLogWriterPlugin

@Inject
public KafkaLogWriterPlugin(CConfiguration cConfig,
                           FileMetaDataManager fileMetaDataManager,
                           LocationFactory locationFactory,
                           CheckpointManagerFactory checkpointManagerFactory)
                     throws Exception

Throws:

Exception

Method Detail

init

public void init(Set<Integer> partitions)

Description copied from interface: KafkaLogProcessor

Called when the leader partitions change.

Parameters:

partitions - new leader partitions.

doProcess

public void doProcess(KafkaLogEvent event)

Description copied from class: AbstractKafkaLogProcessor

doProcess method will be called if the event is not already processed.

Specified by:

doProcess in class AbstractKafkaLogProcessor

Parameters:

event - KafkaLogEvent

stop

public void stop()

Description copied from interface: KafkaLogProcessor

Called to stop processing for the current set of kafka partitions. Stop can be called before init.

getCheckpoint

public Checkpoint getCheckpoint(int partition)

Description copied from interface: KafkaLogProcessor

Get the checkpoint offset for a given partition. This will be used to figure out the lowest offset to read from for any given partition across multiple plugins. If the plugin doesn't care about check pointing offset the implementations can just return -1;

Parameters:

partition - partition number in kafka

Returns:

checkpoint offset