cn.leancloud.kafka.consumer

Class LcKafkaConsumer<K,V>

java.lang.Object

cn.leancloud.kafka.consumer.LcKafkaConsumer<K,V>

Type Parameters:

K - the type of key for records consumed from Kafka

V - the type of value for records consumed from Kafka

All Implemented Interfaces:

Closeable, AutoCloseable

public final class LcKafkaConsumer<K,V>
extends Object
implements Closeable

LcKafkaConsumer is a wrapper over Consumer. It will use Consumer to consume records from Kafka broker.

With LcKafkaConsumer, you can subscribe to several topics and handle all the records from these topics in a dedicated thread pool without warring polling timeout or session timeout due to the polling thread spend too much time on process records and failed to poll broker at least once within max.poll.interval.ms.

All the public methods in LcKafkaConsumer is thread safe.

Method Summary

Modifier and Type	Method and Description
void	close()
Map<org.apache.kafka.common.MetricName,? extends org.apache.kafka.common.Metric>	metrics() Get the metrics kept by the consumer
CompletableFuture<UnsubscribedStatus>	subscribe(Collection<String> topics) Subscribe some Kafka topics to consume records from them.
CompletableFuture<UnsubscribedStatus>	subscribe(Pattern pattern) Subscribe to all topics matching specified pattern to get dynamically assigned partitions.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

subscribe

public CompletableFuture<UnsubscribedStatus> subscribe(Collection<String> topics)

Subscribe some Kafka topics to consume records from them.

Parameters:

topics - the topics to consume.

Returns:

a CompletableFuture which will be complete when the internal KafkaConsumer unsubscribed all the topics that have subscribed to due to an error occurred or close() was called. If it is due to an error occurred, this LcKafkaConsumer will be closed before the returned CompletableFuture complete.

Throws:

IllegalStateException - if this LcKafkaConsumer has closed or subscribed to some topics

NullPointerException - if the input topics is null

IllegalArgumentException - if the input topics is empty or contains null or empty topic

subscribe

public CompletableFuture<UnsubscribedStatus> subscribe(Pattern pattern)

Subscribe to all topics matching specified pattern to get dynamically assigned partitions. The pattern matching will be done periodically against all topics existing at the time of check. This can be controlled through the metadata.max.age.ms configuration: by lowering the max metadata age, the consumer will refresh metadata more often and check for matching topics.

Parameters:

pattern - Pattern to subscribe to.

Returns:

a CompletableFuture which will be complete when the internal KafkaConsumer unsubscribed all the topics that have subscribed to due to an error occurred or close() was called. If it is due to an error occurred, this LcKafkaConsumer will be closed before the returned CompletableFuture complete.

Throws:

IllegalStateException - if this LcKafkaConsumer has closed or subscribed to some topics

NullPointerException - if the input pattern is null

metrics

public Map<org.apache.kafka.common.MetricName,? extends org.apache.kafka.common.Metric> metrics()

Get the metrics kept by the consumer

Returns:

the metrics kept by the consumer

close

public void close()

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable