org.apache.hadoop.resourceestimator.solver.preprocess

Class SolverPreprocessor

java.lang.Object

org.apache.hadoop.resourceestimator.solver.preprocess.SolverPreprocessor

public class SolverPreprocessor
extends Object

Common preprocessing functions for Solver.

Constructor Summary

Constructors

Constructor and Description
SolverPreprocessor()

Method Summary

Methods

Modifier and Type	Method and Description
List<ResourceSkyline>	aggregateSkylines(Map<RecurrenceId,List<ResourceSkyline>> jobHistory, int minJobRuns) Aggregate all job's ResourceSkylines in the one run of recurring pipeline, and return the aggregated ResourceSkylines in different runs.
int[]	getDiscreteSkyline(org.apache.hadoop.yarn.server.resourcemanager.reservation.RLESparseResourceAllocation skyList, int timeInterval, long containerMemAlloc, int jobLen) Discretize job's lifespan into intervals, and return the number of containers used by the job within each interval.
long	getResourceVector(org.apache.hadoop.yarn.server.resourcemanager.reservation.RLESparseResourceAllocation skyList, int index, long containerMemAlloc) Return the multi-dimension resource vector consumed by the job at specified time.
ResourceSkyline	mergeSkyline(List<ResourceSkyline> resourceSkylines) Merge different jobs' resource skylines into one within the same pipeline.
void	validate(Map<RecurrenceId,List<ResourceSkyline>> jobHistory, int timeInterval) Check if Solver's input parameters are valid.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SolverPreprocessor

public SolverPreprocessor()

Method Detail

validate

public final void validate(Map<RecurrenceId,List<ResourceSkyline>> jobHistory,
            int timeInterval)
                    throws InvalidInputException

Check if Solver's input parameters are valid.

Parameters:

jobHistory - the history ResourceSkylines of the recurring pipeline job.

timeInterval - the time interval which is used to discretize the history ResourceSkylines.

Throws:

InvalidInputException - if: (1) jobHistory is null; (2) jobHistory is empty; (3) timeout is non-positive; (4) timeInterval is non-positive;

getResourceVector

public final long getResourceVector(org.apache.hadoop.yarn.server.resourcemanager.reservation.RLESparseResourceAllocation skyList,
                     int index,
                     long containerMemAlloc)

Return the multi-dimension resource vector consumed by the job at specified time.

Parameters:

skyList - the list of Resources used by the job.

index - the discretized time index.

containerMemAlloc - the multi-dimension resource vector allocated to one container.

Returns:

the multi-dimension resource vector consumed by the job.

getDiscreteSkyline

public final int[] getDiscreteSkyline(org.apache.hadoop.yarn.server.resourcemanager.reservation.RLESparseResourceAllocation skyList,
                       int timeInterval,
                       long containerMemAlloc,
                       int jobLen)

Discretize job's lifespan into intervals, and return the number of containers used by the job within each interval.

Note that here we assume all containers allocated to the job have the same Resource. This is due to the limit of RLESparseResourceAllocation.

Parameters:

skyList - the list of Resources used by the job.

timeInterval - the time interval used to discretize the job's lifespan.

containerMemAlloc - the amount of memory allocated to each container.

jobLen - the duration of the job.

Returns:

the number of containers allocated to the job within discretized time intervals.

mergeSkyline

public final ResourceSkyline mergeSkyline(List<ResourceSkyline> resourceSkylines)

Merge different jobs' resource skylines into one within the same pipeline.

Parameters:

resourceSkylines - different jobs' resource skylines within the same pipeline.

Returns:

an aggregated resource skyline for the pipeline.

aggregateSkylines

public final List<ResourceSkyline> aggregateSkylines(Map<RecurrenceId,List<ResourceSkyline>> jobHistory,
                                      int minJobRuns)
                                              throws InvalidInputException

Aggregate all job's ResourceSkylines in the one run of recurring pipeline, and return the aggregated ResourceSkylines in different runs.

Parameters:

jobHistory - the history ResourceSkyline of the recurring pipeline job.

minJobRuns - the minimum number of job runs required to run the solver.

Returns:

the aggregated ResourceSkylines in different runs.

Throws:

InvalidInputException - if: (1) job submission time parsing fails; (2) jobHistory has less job runs than the minimum requirement;