org.apache.flink.streaming.connectors.kafka.internal

Class KafkaConsumerThread

java.lang.Object

java.lang.Thread

org.apache.flink.streaming.connectors.kafka.internal.KafkaConsumerThread

All Implemented Interfaces:

Runnable

@Internal
public class KafkaConsumerThread
extends Thread

The thread the runs the KafkaConsumer, connecting to the brokers and polling records. The thread pushes the data into a Handover to be picked up by the fetcher that will deserialize and emit the records.

IMPORTANT: This thread must not be interrupted when attempting to shut it down. The Kafka consumer code was found to not always handle interrupts well, and to even deadlock in certain situations.

Implementation Note: This code is written to be reusable in later versions of the KafkaConsumer. Because Kafka is not maintaining binary compatibility, we use a "call bridge" as an indirection to the KafkaConsumer calls that change signature.

Nested Class Summary

Nested classes/interfaces inherited from class java.lang.Thread

Thread.State, Thread.UncaughtExceptionHandler

Field Summary

Fields inherited from class java.lang.Thread

MAX_PRIORITY, MIN_PRIORITY, NORM_PRIORITY

Constructor Summary

Constructors

Constructor and Description

KafkaConsumerThread(org.slf4j.Logger log, Handover handover, Properties kafkaProperties, org.apache.flink.streaming.connectors.kafka.internals.ClosableBlockingQueue<org.apache.flink.streaming.connectors.kafka.internals.KafkaTopicPartitionState<org.apache.kafka.common.TopicPartition>> unassignedPartitionsQueue, KafkaConsumerCallBridge consumerCallBridge, String threadName, long pollTimeout, boolean useMetrics, org.apache.flink.metrics.MetricGroup consumerMetricGroup, org.apache.flink.metrics.MetricGroup subtaskMetricGroup)

Method Summary

Modifier and Type	Method and Description
void	run()
void	shutdown() Shuts this thread down, waking up the thread gracefully if blocked (without Thread.interrupt() calls).

Methods inherited from class java.lang.Thread

activeCount, checkAccess, clone, countStackFrames, currentThread, destroy, dumpStack, enumerate, getAllStackTraces, getContextClassLoader, getDefaultUncaughtExceptionHandler, getId, getName, getPriority, getStackTrace, getState, getThreadGroup, getUncaughtExceptionHandler, holdsLock, interrupt, interrupted, isAlive, isDaemon, isInterrupted, join, join, join, resume, setContextClassLoader, setDaemon, setDefaultUncaughtExceptionHandler, setName, setPriority, setUncaughtExceptionHandler, sleep, sleep, start, stop, stop, suspend, toString, yield

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

KafkaConsumerThread

public KafkaConsumerThread(org.slf4j.Logger log,
                           Handover handover,
                           Properties kafkaProperties,
                           org.apache.flink.streaming.connectors.kafka.internals.ClosableBlockingQueue<org.apache.flink.streaming.connectors.kafka.internals.KafkaTopicPartitionState<org.apache.kafka.common.TopicPartition>> unassignedPartitionsQueue,
                           KafkaConsumerCallBridge consumerCallBridge,
                           String threadName,
                           long pollTimeout,
                           boolean useMetrics,
                           org.apache.flink.metrics.MetricGroup consumerMetricGroup,
                           org.apache.flink.metrics.MetricGroup subtaskMetricGroup)

Method Detail

run

public void run()

Specified by:

run in interface Runnable

Overrides:

run in class Thread

shutdown

public void shutdown()

Shuts this thread down, waking up the thread gracefully if blocked (without Thread.interrupt() calls).