org.apache.samza.autoscaling.deployer

Class ConfigManager

java.lang.Object

org.apache.samza.autoscaling.deployer.ConfigManager

public class ConfigManager
extends java.lang.Object

This class is a separate module that runs along side with a job, and handles all config changes submitted to a job after the bootstrap of the job. All config changes are written to the coordinator stream using the @Link{CoordinatorStreamWriter}. The way this class works is that it reads all messages with type "set-config" written to the coordinator stream after the bootstrap of the job, and it handles the messages accordingly. The current configuration changes it handles are 1. changing the number of containers of a job 2. setting the server url for the first time (in order to get the JobModel). In order to use this class the run() method should be called to react to changes, or call the start(), processConfigMessages(), and stop() function instead. Additionally, you have to add the following configurations to the config file: yarn.rm.address=localhost //the ip of the resource manager in yarn yarn.rm.port=8088 //the port of the resource manager http server Additionally, the config manger will periodically poll the coordinator stream to see if there are any new messages. This period is set to 100 ms by default. However, it can be configured by adding the following property to the input config file. configManager.polling.interval=< polling interval >

Constructor Summary

Constructors

Constructor and Description
ConfigManager(org.apache.samza.config.Config config)

Method Summary

Methods

Modifier and Type	Method and Description
java.lang.String	getCoordinatorServerURL() Gets the current value of the server URL that the job coordinator is serving the job model on.
int	getCurrentNumContainers() This method returns the number of containers in the job.
int	getCurrentNumTasks() This method returns the number of tasks in the job.
static void	main(java.lang.String[] args) Main function for using the Config Manager.
void	processConfigMessages() This function reads all the messages with "set-config" type added to the coordinator stream since the last time the method was invoked
void	run() This method is an infinite loop that periodically checks if there are any new messages in the job coordinator stream, and reads them if they exist.
void	start() Starts the system by starting the consumer
void	stop() stops the consumer making the system ready to stop

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ConfigManager

public ConfigManager(org.apache.samza.config.Config config)

Method Detail

run

public void run()

This method is an infinite loop that periodically checks if there are any new messages in the job coordinator stream, and reads them if they exist. Then it reacts accordingly based on the configuration that is being set. The method the calls the start() method to initialized the system, runs in a infinite loop, and calls the stop() method at the end to stop the consumer and the system

start

public void start()

Starts the system by starting the consumer

stop

public void stop()

stops the consumer making the system ready to stop

processConfigMessages

public void processConfigMessages()

This function reads all the messages with "set-config" type added to the coordinator stream since the last time the method was invoked

getCurrentNumTasks

public int getCurrentNumTasks()

This method returns the number of tasks in the job. It works by querying the server, and getting the job model. Then it extracts the number of tasks from the job model

Returns:

current number of tasks in the job

getCurrentNumContainers

public int getCurrentNumContainers()

This method returns the number of containers in the job. It works by querying the server, and getting the job model. Then it extracts the number of containers from the job model

Returns:

current number of containers in the job

getCoordinatorServerURL

public java.lang.String getCoordinatorServerURL()

Gets the current value of the server URL that the job coordinator is serving the job model on.

Returns:

the current server URL. If null, it means the job has not set the server yet.

main

public static void main(java.lang.String[] args)

Main function for using the Config Manager. The main function starts a Config Manager, and reacts to all messages thereafter In order for this module to run, you have to add the following configurations to the config file: yarn.rm.address=localhost //the ip of the resource manager in yarn yarn.rm.port=8088 //the port of the resource manager http server Additionally, the config manger will periodically poll the coordinator stream to see if there are any new messages. This period is set to 100 ms by default. However, it can be configured by adding the following property to the input config file. configManager.polling.interval= < polling interval > To run the code use the following command: {path to samza deployment}/samza/bin/run-config-manager.sh --config-factory={config-factory} --config-path={path to config file of a job}

Parameters:

args - input arguments for running ConfigManager.