public class RmJob extends Object implements SchedConstants, IRmJob
| Modifier and Type | Class and Description |
|---|---|
static class |
RmJob.ShareByInvestmentSorter |
SchedConstants.EvictionPolicy, SchedConstants.Policy| Modifier and Type | Field and Description |
|---|---|
protected boolean |
arbitrary_process |
protected HashMap<Share,Share> |
assignedShares |
protected boolean |
completed |
protected int |
ducc_epoch |
protected org.apache.uima.ducc.transport.event.common.IDuccTypes.DuccType |
ducc_type |
protected org.apache.uima.ducc.common.utils.id.DuccId |
id |
protected boolean |
init_wait |
protected boolean |
is_reservation |
protected int |
job_cap |
protected Properties |
jobprops |
protected int |
max_shares |
protected int |
memory |
protected String |
name |
protected int |
nquestions |
protected int |
nquestions_remaining |
protected int |
orchestrator_epoch |
protected HashMap<Share,Share> |
pendingRemoves |
protected HashMap<Share,Share> |
pendingShares |
protected int |
pure_fair_share |
protected String |
reason |
protected HashMap<Share,Share> |
recoveredShares |
protected ResourceClass |
resource_class |
protected String |
resource_class_name |
protected int |
rm_rate |
protected int |
share_cap |
protected int |
share_order |
protected long |
submit_time |
protected int |
threads |
protected double |
time_per_item |
protected int |
total_assigned |
protected User |
user |
protected int |
user_priority |
protected String |
username |
COMPONENT_NAME, DEFAULT_INIT_STABILITY_COUNT, DEFAULT_INSTANCES, DEFAULT_MAX_INSTANCES, DEFAULT_MAX_PROCESSES, DEFAULT_NODE_METRICS_RATE, DEFAULT_OR_PUBLISH_RATE, DEFAULT_PRIORITY, DEFAULT_PROCESSES, DEFAULT_RM_PUBLISHING_SLOP, DEFAULT_SCHEDULING_RATE, DEFAULT_SCHEDULING_RATIO, DEFAULT_SHARE_WEIGHT, DEFAULT_STABILITY_COUNT| Modifier | Constructor and Description |
|---|---|
protected |
RmJob() |
|
RmJob(org.apache.uima.ducc.common.utils.id.DuccId id) |
| Modifier and Type | Method and Description |
|---|---|
void |
assignShare(Share s)
Scheduler found us a new toy, make it pending until it's given to job manager.
|
int |
calculateCap() |
void |
cancelPending(Share s)
This share is being donated to someone more needy than I - see defrag code in NodepoolScheduler
|
boolean |
canUseBonus(int order)
Can I use more 1 more share of this size?
UIMA-4065
|
void |
clearPendingRemoves()
And finally, dump the pending shrinkage.
|
void |
clearRecoveredShares()
Recovery complete, clear the share map
|
void |
clearShares()
Before each scheduling epoch, clear the counting from the last time.
|
static int |
compareInvestment(Share s1,
Share s2) |
int |
countNShares()
Find number of nShares (virtual shares) this machine has assigned already.
|
int |
countNSharesGiven() |
int |
countNSharesLost()
Number of N-shares I'm losing.
|
int |
countOccupancy() |
int |
countQSharesGiven() |
int |
countTotalAssignments()
Non-preemptable, need to know total every assigned, in case one of them dies, must be careful
not to reassign it.
|
void |
defer(String reason)
Can't schedule non-preemptable yet, but maybe later
|
boolean |
equals(Object obj) |
boolean |
exceedsFairShareCap() |
Comparator<org.apache.uima.ducc.rm.scheduler.IEntity> |
getApportionmentSorter() |
HashMap<Share,Share> |
getAssignedShares()
Officially allocated shares assigned to this job which are known to be in use.
|
String |
getClassName() |
org.apache.uima.ducc.transport.event.common.IDuccTypes.DuccType |
getDuccType() |
long |
getFriendlyId() |
int[] |
getGivenByOrder() |
static String |
getHeader() |
org.apache.uima.ducc.common.utils.id.DuccId |
getId() |
int |
getJobCap()
This returns the largest number that can actually be used, which will be either the
share cap itself, or nProcess / nThreads, in quantum shares.
|
Map<Machine,Machine> |
getMachines() |
int |
getMaxShares() |
int |
getMemory() |
String |
getName() |
HashMap<Share,Share> |
getPendingRemoves()
Return the reclaimed shares.
|
HashMap<Share,Share> |
getPendingShares()
Newly allocated shares that have not been dispatched.
|
int |
getPureFairShare() |
String |
getReason() |
HashMap<Share,Share> |
getRecoveredShares()
Shares recovered from the OR during job recovery.
|
String |
getRefusalReason() |
ResourceClass |
getResourceClass() |
SchedConstants.Policy |
getSchedulingPolicy() |
int |
getSchedulingPriority() |
int |
getShareCap() |
int |
getShareOrder() |
Map<Machine,Map<Share,Share>> |
getSharesByMachine()
For defrag, we want to know what machines a job is running on.
|
int |
getShareWeight() |
String |
getShortType() |
long |
getTimestamp() |
User |
getUser() |
String |
getUserName() |
int |
getUserPriority() |
int[] |
getWantedByOrder() |
int |
hashCode() |
void |
init()
read the props file and set inital values for internal fields
|
void |
initJobCap()
This returns the largest number that can actually be used, which will be either the
share cap itself, or nProcess / nThreads, in N shares.
|
void |
initWantedByOrder(ResourceClass unused) |
boolean |
isArbitraryProcess() |
boolean |
isCompleted() |
boolean |
isDeferred() |
boolean |
isDormant() |
boolean |
isExpanded()
Do I have un-dispatched shares?
|
boolean |
isInitialized()
Is at least one of my current shares initialized?
|
boolean |
isPendingShare(Share s) |
boolean |
isRefused() |
boolean |
isReservation() |
boolean |
isRunning() |
boolean |
isService() |
boolean |
isShrunk()
Waiting for somebody to deal with my shrinkage?
|
boolean |
isShrunken() |
boolean |
isStable() |
void |
markComplete()
For preemptable, must remember the job completed so we don't accidentally reexpand it.
|
int |
nQuestions()
Number of questions submitted, all must be answered.
|
int |
nQuestionsRemaining()
Number of questions still to be answered.
|
int |
nThreads() |
String |
printShares()
Logging and debugging - nice to know what my assigned shares are.
|
HashMap<Share,Share> |
promoteShares()
We're dispatching, move machines to active list, and clear pending list.
|
int |
queryDemand()
For queries - how many processes do I want in a perfect world?
|
void |
recoverShare(Share s)
Job recovery: OR reports this share as one it already knew about.
|
void |
refuse(String refusal)
Can't schedudle this nohow.
|
void |
removeAllShares()
Clear the job, it has no resources of any form.
|
void |
removeShare(Share share)
I've shrunk or this share has nothing left to do.
|
void |
setArbitraryProcess() |
void |
setClassName(String class_name) |
void |
setDuccType(org.apache.uima.ducc.transport.event.common.IDuccTypes.DuccType type) |
void |
setGivenByOrder(int[] gbo) |
void |
setId(org.apache.uima.ducc.common.utils.id.DuccId id) |
boolean |
setInitWait(boolean is_running)
The matrix:
IW = am I in initializaton wait
RU = am I runnable
Resched = do we need to reschedule
Reset = set IW to this
x = don't care
IW RU Resched Reset
T T T F
T F F T
F T F F
F F x x
So resched = IR & RU
IW = !RU
We return resched so caller knows to tickle the scheduler
|
void |
setJobName(String name) |
void |
setMaxShares(int s) |
void |
setMemory(int memory) |
void |
setNQuestions(int allq,
int remainingq,
double time_per_item) |
void |
setPureFairShare(int pfs)
public void addQShares(int s)
{
this.shares_given += ( s / share_order ) ; // convert to N-shares == processes
}
|
void |
setReason(String reason) |
void |
setReservation() |
void |
setResourceClass(ResourceClass cl)
Save ref to the class we are in, and init class-based structures.
|
void |
setShareCap(int cap)
During the scheduling algorithm we want to track some things by userid.
|
void |
setShareOrder(int s)
Scheduler looks at job memory and decides what its share order is.
|
void |
setThreads(int th) |
void |
setTimestamp(long t) |
void |
setUser(User u) |
void |
setUserName(String n) |
void |
setUserPriority(int p) |
void |
shrinkBy(int howmany)
This is an investment shrink.
|
int |
shrinkByInvestment(int shares,
int order,
boolean force,
org.apache.uima.ducc.rm.scheduler.NodePool nodepool)
Shrink by 'shares' from machines of largest order starting from 'order' and decreasing.
|
void |
shrinkByOne(Share share)
Remove a process from this job
|
int |
shrinkByOrderByMachine(int shares,
int order,
boolean force,
org.apache.uima.ducc.rm.scheduler.NodePool nodepool)
Shrink by 'shares' from machines of largest order starting from 'order' and decreasing.
|
String |
toString() |
String |
toStringWithHeader() |
void |
undefer() |
protected org.apache.uima.ducc.common.utils.id.DuccId id
protected org.apache.uima.ducc.transport.event.common.IDuccTypes.DuccType ducc_type
protected boolean arbitrary_process
protected String name
protected String resource_class_name
protected ResourceClass resource_class
protected int user_priority
protected int max_shares
protected boolean is_reservation
protected int threads
protected int memory
protected int nquestions
protected int nquestions_remaining
protected double time_per_item
protected int share_order
protected int share_cap
protected int job_cap
protected int pure_fair_share
protected long submit_time
protected String username
protected User user
protected int total_assigned
protected boolean init_wait
protected boolean completed
protected int orchestrator_epoch
protected int rm_rate
protected int ducc_epoch
protected Properties jobprops
protected String reason
protected RmJob()
public RmJob(org.apache.uima.ducc.common.utils.id.DuccId id)
public void init()
public org.apache.uima.ducc.common.utils.id.DuccId getId()
public void setId(org.apache.uima.ducc.common.utils.id.DuccId id)
public long getFriendlyId()
getFriendlyId in interface IRmJobpublic void setJobName(String name)
setJobName in interface IRmJobpublic void setReservation()
setReservation in interface IRmJobpublic boolean isReservation()
isReservation in interface IRmJobpublic void markComplete()
markComplete in interface IRmJobpublic boolean isCompleted()
isCompleted in interface IRmJobpublic boolean setInitWait(boolean is_running)
setInitWait in interface IRmJobpublic void setResourceClass(ResourceClass cl)
setResourceClass in interface IRmJobpublic int nQuestions()
nQuestions in interface IRmJobpublic void setNQuestions(int allq,
int remainingq,
double time_per_item)
setNQuestions in interface IRmJobpublic int nQuestionsRemaining()
nQuestionsRemaining in interface IRmJobpublic Map<Machine,Map<Share,Share>> getSharesByMachine()
IRmJobgetSharesByMachine in interface IRmJobpublic Map<Machine,Machine> getMachines()
getMachines in interface IRmJobpublic int queryDemand()
IRmJobqueryDemand in interface IRmJobpublic void clearShares()
clearShares in interface IRmJobpublic void setPureFairShare(int pfs)
setPureFairShare in interface IRmJobpublic int getPureFairShare()
getPureFairShare in interface IRmJobpublic int[] getGivenByOrder()
public void setGivenByOrder(int[] gbo)
public int getShareWeight()
public void initWantedByOrder(ResourceClass unused)
public int[] getWantedByOrder()
public int calculateCap()
public int countOccupancy()
countOccupancy in interface IRmJobpublic int countNSharesGiven()
countNSharesGiven in interface IRmJobpublic int countQSharesGiven()
countQSharesGiven in interface IRmJobpublic int countNSharesLost()
countNSharesLost in interface IRmJobpublic boolean canUseBonus(int order)
order - The size of the available share. Must be an exact match because the
offerer has already done all reasonable splitting and will have a better
use for it if I can't take it.
The decision is based on the wbo/gbo arrays that the offer has been building up
just before asking this question.public HashMap<Share,Share> getAssignedShares()
getAssignedShares in interface IRmJobpublic HashMap<Share,Share> getRecoveredShares()
getRecoveredShares in interface IRmJobpublic HashMap<Share,Share> getPendingShares()
getPendingShares in interface IRmJobpublic HashMap<Share,Share> promoteShares()
promoteShares in interface IRmJobpublic void cancelPending(Share s)
cancelPending in interface IRmJobpublic void assignShare(Share s)
assignShare in interface IRmJobpublic int countTotalAssignments()
countTotalAssignments in interface IRmJobpublic void recoverShare(Share s)
recoverShare in interface IRmJobpublic boolean isPendingShare(Share s)
isPendingShare in interface IRmJobpublic boolean isExpanded()
isExpanded in interface IRmJobpublic boolean isShrunken()
isShrunken in interface IRmJobpublic void removeAllShares()
IRmJobremoveAllShares in interface IRmJobpublic void removeShare(Share share)
removeShare in interface IRmJobpublic void shrinkByOne(Share share)
shrinkByOne in interface IRmJobpublic int shrinkByOrderByMachine(int shares,
int order,
boolean force,
org.apache.uima.ducc.rm.scheduler.NodePool nodepool)
shrinkByOrderByMachine in interface IRmJobshares - - number of N-shares that are wantedorder - - N - try to free up space for shares of this size.force - - When evicting for non-preemptables, we may need to free the requested
shares even if it puts us over our "fair" count. If this happens
we'll end up "sliding" onto other machines (eventually).nodepool - - only interested in shares from this nodepool.
So when this is called, somebody needs (shares*order) shares, given out in chunks of
(order).public int shrinkByInvestment(int shares,
int order,
boolean force,
org.apache.uima.ducc.rm.scheduler.NodePool nodepool)
shrinkByInvestment in interface IRmJobshares - - number of N-shares that are wantedorder - - N - try to free up space for shares of this size.force - - When evicting for non-preemptables, we may need to free the requested
shares even if it puts us over our "fair" count. If this happens
we'll end up "sliding" onto other machines (eventually).nodepool - - only interested in shares from this nodepool.
So when this is called, somebody needs (shares*order) shares, given out in chunks of
(order).public void shrinkBy(int howmany)
public boolean isShrunk()
public HashMap<Share,Share> getPendingRemoves()
getPendingRemoves in interface IRmJobpublic void clearPendingRemoves()
clearPendingRemoves in interface IRmJobpublic void clearRecoveredShares()
clearRecoveredShares in interface IRmJobpublic int countNShares()
countNShares in interface IRmJobpublic void refuse(String refusal)
IRmJobpublic void defer(String reason)
IRmJobpublic boolean isDeferred()
isDeferred in interface IRmJobpublic String getRefusalReason()
getRefusalReason in interface IRmJobpublic void setShareOrder(int s)
IRmJobsetShareOrder in interface IRmJobpublic int getShareOrder()
getShareOrder in interface IRmJobpublic void setShareCap(int cap)
public int getShareCap()
public void initJobCap()
initJobCap in interface IRmJobpublic int getJobCap()
IRmJobpublic int getMaxShares()
getMaxShares in interface IRmJobpublic void setMaxShares(int s)
setMaxShares in interface IRmJobpublic boolean isRunning()
public String getUserName()
getUserName in interface IRmJobpublic void setUserName(String n)
setUserName in interface IRmJobpublic long getTimestamp()
getTimestamp in interface IRmJobpublic void setTimestamp(long t)
setTimestamp in interface IRmJobpublic int getUserPriority()
getUserPriority in interface IRmJobpublic void setUserPriority(int p)
setUserPriority in interface IRmJobpublic String getClassName()
getClassName in interface IRmJobpublic void setClassName(String class_name)
setClassName in interface IRmJobpublic int getSchedulingPriority()
getSchedulingPriority in interface IRmJobpublic SchedConstants.Policy getSchedulingPolicy()
getSchedulingPolicy in interface IRmJobpublic ResourceClass getResourceClass()
getResourceClass in interface IRmJobpublic boolean exceedsFairShareCap()
exceedsFairShareCap in interface IRmJobpublic void setThreads(int th)
setThreads in interface IRmJobpublic void setDuccType(org.apache.uima.ducc.transport.event.common.IDuccTypes.DuccType type)
setDuccType in interface IRmJobpublic org.apache.uima.ducc.transport.event.common.IDuccTypes.DuccType getDuccType()
getDuccType in interface IRmJobpublic void setArbitraryProcess()
setArbitraryProcess in interface IRmJobpublic boolean isArbitraryProcess()
isArbitraryProcess in interface IRmJobpublic boolean isInitialized()
isInitialized in interface IRmJobpublic String printShares()
printShares in interface IRmJobpublic String getShortType()
getShortType in interface IRmJobpublic static String getHeader()
public String toStringWithHeader()
public Comparator<org.apache.uima.ducc.rm.scheduler.IEntity> getApportionmentSorter()
Copyright © 2012–2015 The Apache Software Foundation. All rights reserved.