co.cask.cdap.logging.kafka

Class KafkaConsumer

java.lang.Object

co.cask.cdap.logging.kafka.KafkaConsumer

All Implemented Interfaces:

Closeable, AutoCloseable

public final class KafkaConsumer
extends Object
implements Closeable

Kafka consumer that listens on a topic/partition and retrieves messages. This class is thread-safe.

Field Summary

Fields

Modifier and Type	Field and Description
static int	BUFFER_SIZE_BYTES
static long	EARLIEST_OFFSET
static long	LATEST_OFFSET
static int	TIMEOUT_MS

Constructor Summary

Constructors

Constructor and Description
KafkaConsumer(List<LoggingConfiguration.KafkaHost> seedBrokers, String topic, int partition, int fetchTimeoutMs) Creates a KafkaConsumer with initial set of seed brokers, topic and partition.

Method Summary

Methods

Modifier and Type	Method and Description
void	close()
int	fetchMessages(long offset, Callback callback) Fetches Kafka messages from an offset.
long	fetchOffsetBefore(long timeMillis) Fetch offset before given time.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

EARLIEST_OFFSET

public static final long EARLIEST_OFFSET

See Also:

Constant Field Values

LATEST_OFFSET

public static final long LATEST_OFFSET

See Also:

Constant Field Values

BUFFER_SIZE_BYTES

public static final int BUFFER_SIZE_BYTES

See Also:

Constant Field Values

TIMEOUT_MS

public static final int TIMEOUT_MS

See Also:

Constant Field Values

Constructor Detail

KafkaConsumer

public KafkaConsumer(List<LoggingConfiguration.KafkaHost> seedBrokers,
             String topic,
             int partition,
             int fetchTimeoutMs)

Creates a KafkaConsumer with initial set of seed brokers, topic and partition.

Parameters:

seedBrokers - list of seed brokers

topic - Kafka topic to subscribe to

partition - topic partition to subscribe to

fetchTimeoutMs - timeout for a Kafka fetch call

Method Detail

fetchMessages

public int fetchMessages(long offset,
                Callback callback)
                  throws kafka.common.OffsetOutOfRangeException

Fetches Kafka messages from an offset.

Parameters:

offset - message offset to start.

callback - callback to handle the messages fetched.

Returns:

number of messages fetched.

Throws:

kafka.common.OffsetOutOfRangeException

fetchOffsetBefore

public long fetchOffsetBefore(long timeMillis)

Fetch offset before given time.

Parameters:

timeMillis - offset to fetch before timeMillis.

Returns:

Kafka message offset

close

public void close()
           throws IOException

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Throws:

IOException