Package jasima.core.experiment

Class OCBAExperiment

  • All Implemented Interfaces:
    Notifier<Experiment,​Experiment.ExperimentEvent>, ValueStore, Serializable, Cloneable
    public class OCBAExperiment
extends FullFactorialExperiment

    Purpose of this class is to find the best configuration/parameterization of a base experiment (subject to random effects) using the Optimal Computing Budget Allocation (OCBA) method. In contrast to simply running a FullFactorialExperiment performing a fixed number of replications for each configuration, this class aims at intelligently distributing a given budget of base experiment runs in order to maximize the probability of actually selecting the best configuration (Probability of Correct Selection, PCS).

    Implements the OCBA method as described in Chen2000: Chen, C. H., J. Lin, E. Yücesan, and S. E. Chick, "Simulation Budget Allocation for Further Enhancing the Efficiency of Ordinal Optimization," Journal of Discrete Event Dynamic Systems: Theory and Applications, Vol. 10, pp. 251-270, July 2000.

    First minReplicationsPerConfiguration replications (default: 5) are performed for each configuration. Later on runs are allocated dynamically using OCBA. The total budget is given by: getNumReplications() (default: 10)*<number of configurations>

    To use this class at least the name of the objective value ( setObjective(String)) and whether this objective is to be maximized or minimized (setMaximize()) have to be set.

    Each iteration of the allocation algorithm allocates more than a single run in order to benefit from parallelization.

    A usage example is given below. It selects the best of two dispatching rules of a dynamic job shop scenario. 

     // create and configure base experiment
 HolthausExperiment he = new HolthausExperiment();
 he.setUtilLevel(0.9);
 he.addShopListener(new BasicJobStatCollector());
 
 // create OCBA experiment
 OCBAExperiment ocbaExperiment = new OCBAExperiment();
 ocbaExperiment.setInitialSeed(23);
 
 // set base experiment to use
 ocbaExperiment.setBaseExperiment(he);
 
 // define configurations to test
 ocbaExperiment.addFactor("sequencingRule", new SPT().setFinalTieBreaker(new TieBreakerFASFS()));
 ocbaExperiment.addFactor("sequencingRule", new PTPlusWINQPlusNPT().setFinalTieBreaker(new TieBreakerFASFS()));
 
 // define objective function
 ocbaExperiment.setObjective("flowMean");
 ocbaExperiment.setProblemType(ProblemType.MINIMIZE);
 
 // no fixed budget, run until we are pretty sure to have the best
 // configuration
 ocbaExperiment.setNumReplications(0);
 ocbaExperiment.setPcsLevel(0.95);
 
 // optionally produce an Excel file with results and details
 ocbaExperiment.addNotifierListener(new ExcelSaver());
 
 // run
 ocbaExperiment.runExperiment();
 ocbaExperiment.printResults();

    This class implements a basic ranking and selection method. In future versions it would be very helpful to improve its algorithm to better deal with very similar performances of good configurations (such as indifference zone approaches, or using Expected Opportunity Costs). I'm also unsure about the effects if base experiments use common random numbers (which is the default behavior of this class, see FullFactorialExperiment.setCommonRandomNumbers(boolean). In summary, this class is not a fool-proof intelligent allocator of replications, but should provide reasonably good results to be useful. Probably it's also a good starting point for experts in the field to implement (and contribute?) improved algorithms.

    Author:
    Torsten Hildebrandt
    See Also:
    MultipleReplicationExperiment, FullFactorialExperiment, Serialized Form

    • Constructor Detail

      • OCBAExperiment

        public OCBAExperiment()

    • Method Detail

      • calcPCS

        protected double calcPCS()

      • calcPCSPriosPerConfiguration

        protected double[] calcPCSPriosPerConfiguration()

      • ocba

        protected int[] ocba​(int add_budget)
        This subroutine implements the optimal computation budget allocation (OCBA) algorithm presented in Chen et al. (2000) in the J of DEDS. It determines how many additional runs each design should have for the next iteration of simulation.
        Parameters:
        add_budget - The total number of additional replications that can be performed.
        Returns:
        additional number of simulation replication assigned to design i, i=0,1,..,ND-1

      • setMinReplicationsPerConfiguration

        public void setMinReplicationsPerConfiguration​(int minReps)
        Sets the minimum number of replications performed for each configuration. This has to be >=3 or -1 to use the number of available processor cores.
        Parameters:
        minReps - The minimum number of replications per configuration.

      • getMinReplicationsPerConfiguration

        public int getMinReplicationsPerConfiguration()

      • setObjective

        public void setObjective​(String objective)
        Sets the name of the objective which defines "best". This has to be the name of a result produced by the base experiment.
        Parameters:
        objective - Result name to use as the objective function.

      • getObjective

        public String getObjective()

      • setPcsLevel

        public void setPcsLevel​(double pcsLevel)
        Stop using more replications if this level of the probablity of correct selection is reached. This defines a dynamic stopping criterion.
        Parameters:
        pcsLevel - The desired confidence in the results (between 0 and 1).

      • getPcsLevel

        public double getPcsLevel()

      • setDetailedResults

        public void setDetailedResults​(boolean detailedResults)
        Whether to produce detailed results or just basic information of the best configuration.
        Parameters:
        detailedResults - Produce detailed results or not.

      • isDetailedResults

        public boolean isDetailedResults()

      • setNumReplicationsPerConfiguration

        public void setNumReplicationsPerConfiguration​(int numReplications)
        Sets the total budget for each configuration.
        Parameters:
        numReplications - The number of replications to use.

      • getNumReplicationsPerConfiguration

        public int getNumReplicationsPerConfiguration()

      • setProblemType

        public void setProblemType​(OCBAExperiment.ProblemType problemType)
        Sets whether a minimization or maximization experiment should be solved.
        Parameters:
        problemType - Whether minimization or maximization are required.