org.apache.uima.ducc.rm.scheduler
Class NodepoolScheduler

java.lang.Object
  org.apache.uima.ducc.rm.scheduler.NodepoolScheduler

All Implemented Interfaces:

IScheduler, SchedConstants

public class NodepoolScheduler
extends Object
implements IScheduler, SchedConstants

This implementation of IScheduler is the initial implementation of a scheduler using classes.

setClasses

public void setClasses(Map<ResourceClass,ResourceClass> prclasses)

Specified by:

setClasses in interface IScheduler

setNodePool

public void setNodePool(org.apache.uima.ducc.rm.scheduler.NodePool np)

Specified by:

setNodePool in interface IScheduler

setEvictionPolicy

public void setEvictionPolicy(SchedConstants.EvictionPolicy ep)

Specified by:

setEvictionPolicy in interface IScheduler

apportion_qshares

protected void apportion_qshares(List<org.apache.uima.ducc.rm.scheduler.IEntity> entities,
                                 int[] vshares,
                                 int total_shares,
                                 String descr)

countClassShares

protected void countClassShares(org.apache.uima.ducc.rm.scheduler.NodePool np,
                                ArrayList<ResourceClass> rcs)

Count out shares for only the jobs in the ResouceClasses here, and only from the given nodepool.

countUserShares

protected boolean countUserShares(ResourceClass rc)

Count out shares for only the jobs in the ResouceClasses here, and only from the given nodepool.

traverseNodepoolsForCounts

protected void traverseNodepoolsForCounts(org.apache.uima.ducc.rm.scheduler.NodePool np,
                                          ArrayList<ResourceClass> rcs)

Depth-first traversal of the nodepool. Once you get to a leaf, count the shares. This sets an upper-bound on the number of shares a class can have. As you wind back up the tree the counts may be reduced because of competition from jobs in the parent node. By the time we're dont we should have accounted for all jobs and all usable resources. Note how this works: Consider a configuration with two nodepools plus global, A, B, and G. Suppose nodepools A and B have 30 shares each and G only has 10 shares. G can apportion over it's 10, plus the 60 from A and B. So after apporioning over A and B we need to do fair-share over G+A+B to insure that jobs submitted to G are not "cheated" - recall that jobs in this set of classes have the same weight and priority, and thus the same "right" to all the shares. However, allocating a job from class A over the full set of 10+30 shares could over-allocate it. So the cap calculations must be sure never to increase the already-given shares for subpools. Therefore we traverse the FULL SET of classes on every recursion. When calculating caps from apportion_shares the resource classes will have to account for multiple traversals and not over-allocate if a class has already been apportioned from a subpool.

updateNodepools

protected void updateNodepools(org.apache.uima.ducc.rm.scheduler.NodePool np,
                               ArrayList<ResourceClass> rcs)

jobInClass

protected boolean jobInClass(ArrayList<ResourceClass> rcs,
                             IRmJob j)

Is the job submitted to one of these classes?

expandNeedyJobs

protected void expandNeedyJobs(org.apache.uima.ducc.rm.scheduler.NodePool np,
                               ArrayList<ResourceClass> rcs)

Expand jobs from the needy list ahead of other jobs, because we likely stole the shares from older jobs which would otherwise get priority.

traverseNodepoolsForExpansion

protected void traverseNodepoolsForExpansion(org.apache.uima.ducc.rm.scheduler.NodePool np,
                                             ArrayList<ResourceClass> rcs)

whatOfFairShare

protected void whatOfFairShare(ArrayList<ResourceClass> rcs)

howMuchFixed

protected void howMuchFixed(ArrayList<ResourceClass> rcs)

Make sure there are enough shares to allocate either directly, or through preemption, and count them out.

whatOfFixedShare

protected void whatOfFixedShare(ArrayList<ResourceClass> rcs)

All-or-nothing makes this easy. If there are free shares of the right order just assign them. Otherwise the counts will cause evictions in lower-priority code so we just wait.

accountForNonPreemptable

protected void accountForNonPreemptable()

Remove non-preemptable resources from the counts.

accountForFairShare

protected void accountForFairShare()

Remove active shares from the nodepool counts, leaving us with just the free shares in the tables.

findHowMuch

protected void findHowMuch(ArrayList<ResourceClass> rcs)

The second stage - work up all counts globally

doEvictions

protected void doEvictions(org.apache.uima.ducc.rm.scheduler.NodePool nodepool)

findWhatOf

protected void findWhatOf(ArrayList<ResourceClass> rcs)

The third stage - work up all counts globally

schedule

public void schedule(SchedulingUpdate upd)

IScheduler entry point for the fairShare calculation. This implements the easy three step process described at the top of the file.

Specified by:

schedule in interface IScheduler