Package de.jungblut.math.minimize

Class AbstractMiniBatchCostFunction

  • All Implemented Interfaces:
    CostFunction
    Direct Known Subclasses:
    MultilayerPerceptronCostFunction, RBMCostFunction
    public abstract class AbstractMiniBatchCostFunction
extends java.lang.Object
implements CostFunction
    Mini Batch cost function. It features parallel calculation of mini batches and averaging of the error and gradient. In addition you can calculate mini-batches in a stochastic fashion, that means that every called iteration a new mini-batch will be evaluated, its gradient will immediately be returned.

    The resulting costfunction can be minimized with every normal Minimizer. The extending cost functions should be as stateless as possible.
    Author:
    thomas.jungblut

    • Constructor Summary

      Constructors 
      Constructor Description
      AbstractMiniBatchCostFunction​(de.jungblut.math.DoubleVector[] inputMatrix, de.jungblut.math.DoubleVector[] outcomeMatrix, int batchSize, int numThreads)
      An abstract minibatch costfunction.
      AbstractMiniBatchCostFunction​(de.jungblut.math.DoubleVector[] inputMatrix, de.jungblut.math.DoubleVector[] outcomeMatrix, int batchSize, int numThreads, boolean stochastic)
      An abstract minibatch costfunction.

    • Method Summary

      All Methods Instance Methods Abstract Methods Concrete Methods 
      Modifier and Type Method Description
      protected abstract CostGradientTuple evaluateBatch​(de.jungblut.math.DoubleVector theta, de.jungblut.math.DoubleMatrix featureBatch, de.jungblut.math.DoubleMatrix outcomeBatch)
      Evaluate the batch.
      CostGradientTuple evaluateCost​(de.jungblut.math.DoubleVector input)
      Evaluation for the cost function to retrieve cost and gradient.

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Constructor Detail

      • AbstractMiniBatchCostFunction

        public AbstractMiniBatchCostFunction​(de.jungblut.math.DoubleVector[] inputMatrix,
                                     de.jungblut.math.DoubleVector[] outcomeMatrix,
                                     int batchSize,
                                     int numThreads)
        An abstract minibatch costfunction. This trains on all batches and averages their results (gradient&cost).
        Parameters:
        inputMatrix - the data to crunch, bias will be added while calculating the batches.
        outcomeMatrix - the data that denotes the outcome of the features- while supervised learning. Can be null for unsupervised methods.
        batchSize - the batch size to use, 0 denotes full batch learning by default.
        numThreads - the number of threads to use to calculate the batches.

      • AbstractMiniBatchCostFunction

        public AbstractMiniBatchCostFunction​(de.jungblut.math.DoubleVector[] inputMatrix,
                                     de.jungblut.math.DoubleVector[] outcomeMatrix,
                                     int batchSize,
                                     int numThreads,
                                     boolean stochastic)
        An abstract minibatch costfunction.
        Parameters:
        inputMatrix - the data to crunch, bias will be added while calculating the batches.
        outcomeMatrix - the data that denotes the outcome of the features- while supervised learning. Can be null for unsupervised methods.
        batchSize - the batch size to use, 0 denotes full batch learning by default.
        numThreads - the number of threads to use to calculate the batches.
        stochastic - if true the batches will be evaluated stochastically, that means that each iteration the next batch is chosen to evaluate and calculate gradient and cost.

    • Method Detail

      • evaluateCost

        public final CostGradientTuple evaluateCost​(de.jungblut.math.DoubleVector input)
        Description copied from interface: CostFunction
        Evaluation for the cost function to retrieve cost and gradient.
        Specified by:
        evaluateCost in interface CostFunction
        Parameters:
        input - a given input vector
        Returns:
        a tuple consists of J (cost) and a vector X which is the gradient of the input.

      • evaluateBatch

        protected abstract CostGradientTuple evaluateBatch​(de.jungblut.math.DoubleVector theta,
                                                   de.jungblut.math.DoubleMatrix featureBatch,
                                                   de.jungblut.math.DoubleMatrix outcomeBatch)
        Evaluate the batch.
        Parameters:
        theta - the parameters to use.
        featureBatch - the batch matrix as input (already contains a bias!).
        outcomeBatch - the batch matrix denoting the output.
        Returns:
        the cost/gradient tuple usually used when using evaluateCost(DoubleVector).