org.apache.gobblin.service.monitoring

Class KafkaAvroJobStatusMonitor

java.lang.Object

com.google.common.util.concurrent.AbstractIdleService

org.apache.gobblin.runtime.kafka.HighLevelConsumer<byte[],byte[]>

org.apache.gobblin.service.monitoring.KafkaJobStatusMonitor

org.apache.gobblin.service.monitoring.KafkaAvroJobStatusMonitor

All Implemented Interfaces:

com.google.common.util.concurrent.Service

public class KafkaAvroJobStatusMonitor
extends KafkaJobStatusMonitor

A job status monitor for Avro messages. Uses GobblinTrackingEvent schema to parse the messages and calls parseJobStatus(GobblinTrackingEvent) for each received message.

Nested Class Summary

Nested classes/interfaces inherited from interface com.google.common.util.concurrent.Service

com.google.common.util.concurrent.Service.Listener, com.google.common.util.concurrent.Service.State

Field Summary

Fields inherited from class org.apache.gobblin.service.monitoring.KafkaJobStatusMonitor

GET_AND_SET_JOB_STATUS, STATE_STORE_KEY_SEPARATION_CHARACTER, STATE_STORE_TABLE_SUFFIX

Fields inherited from class org.apache.gobblin.runtime.kafka.HighLevelConsumer

config, CONSUMER_CLIENT_FACTORY_CLASS_KEY, DEFAULT_AUTO_COMMIT_VALUE, DEFAULT_OFFSET_COMMIT_NUM_RECORDS_THRESHOLD, DEFAULT_OFFSET_COMMIT_TIME_THRESHOLD_SECS, ENABLE_AUTO_COMMIT_KEY, GROUP_ID_KEY, OFFSET_COMMIT_NUM_RECORDS_THRESHOLD_KEY, OFFSET_COMMIT_TIME_THRESHOLD_SECS_KEY, topic

Constructor Summary

Constructors

Constructor and Description
KafkaAvroJobStatusMonitor(String topic, com.typesafe.config.Config config, int numThreads)

Method Summary

Modifier and Type	Method and Description
protected void	createMetrics()
State	parseJobStatus(byte[] message)

Methods inherited from class org.apache.gobblin.service.monitoring.KafkaJobStatusMonitor

getExecutionIdFromTableName, jobStatusStoreName, jobStatusTableName, jobStatusTableName, processMessage, shutDown, startUp

Methods inherited from class org.apache.gobblin.runtime.kafka.HighLevelConsumer

buildMetricsContextAndMetrics, commitOffsets, createConsumerClient, getGobblinKafkaConsumerClient, getMetricContext, getTagsForMetrics, getTopic, shutdownMetrics

Methods inherited from class com.google.common.util.concurrent.AbstractIdleService

addListener, awaitRunning, awaitRunning, awaitTerminated, awaitTerminated, executor, failureCause, isRunning, serviceName, start, startAndWait, startAsync, state, stop, stopAndWait, stopAsync, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

KafkaAvroJobStatusMonitor

public KafkaAvroJobStatusMonitor(String topic,
                                 com.typesafe.config.Config config,
                                 int numThreads)
                          throws IOException,
                                 ReflectiveOperationException

Throws:

IOException

ReflectiveOperationException

Method Detail

createMetrics

protected void createMetrics()

Overrides:

createMetrics in class KafkaJobStatusMonitor

parseJobStatus

public State parseJobStatus(byte[] message)
                     throws IOException

Specified by:

parseJobStatus in class KafkaJobStatusMonitor

Throws:

IOException