de.jungblut.math.minimize

Class AbstractMiniBatchCostFunction

java.lang.Object

de.jungblut.math.minimize.AbstractMiniBatchCostFunction

All Implemented Interfaces:

CostFunction

Direct Known Subclasses:

MultilayerPerceptronCostFunction, RBMCostFunction

public abstract class AbstractMiniBatchCostFunction
extends Object
implements CostFunction

Mini Batch cost function. It features parallel calculation of mini batches and averaging of the error and gradient. In addition you can calculate mini-batches in a stochastic fashion, that means that every called iteration a new mini-batch will be evaluated, its gradient will immediately be returned.

The resulting costfunction can be minimized with every normal Minimizer. The extending cost functions should be as stateless as possible.

Author:

thomas.jungblut

Constructor Summary

Constructors

Constructor and Description
AbstractMiniBatchCostFunction(de.jungblut.math.DoubleVector[] inputMatrix, de.jungblut.math.DoubleVector[] outcomeMatrix, int batchSize, int numThreads) An abstract minibatch costfunction.
AbstractMiniBatchCostFunction(de.jungblut.math.DoubleVector[] inputMatrix, de.jungblut.math.DoubleVector[] outcomeMatrix, int batchSize, int numThreads, boolean stochastic) An abstract minibatch costfunction.

Method Summary

Modifier and Type	Method and Description
protected abstract CostGradientTuple	evaluateBatch(de.jungblut.math.DoubleVector theta, de.jungblut.math.DoubleMatrix featureBatch, de.jungblut.math.DoubleMatrix outcomeBatch) Evaluate the batch.
CostGradientTuple	evaluateCost(de.jungblut.math.DoubleVector input) Evaluation for the cost function to retrieve cost and gradient.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

AbstractMiniBatchCostFunction

public AbstractMiniBatchCostFunction(de.jungblut.math.DoubleVector[] inputMatrix,
                                     de.jungblut.math.DoubleVector[] outcomeMatrix,
                                     int batchSize,
                                     int numThreads)

An abstract minibatch costfunction. This trains on all batches and averages their results (gradient&cost).

Parameters:

inputMatrix - the data to crunch, bias will be added while calculating the batches.

outcomeMatrix - the data that denotes the outcome of the features- while supervised learning. Can be null for unsupervised methods.

batchSize - the batch size to use, 0 denotes full batch learning by default.

numThreads - the number of threads to use to calculate the batches.

AbstractMiniBatchCostFunction

public AbstractMiniBatchCostFunction(de.jungblut.math.DoubleVector[] inputMatrix,
                                     de.jungblut.math.DoubleVector[] outcomeMatrix,
                                     int batchSize,
                                     int numThreads,
                                     boolean stochastic)

An abstract minibatch costfunction.

Parameters:

inputMatrix - the data to crunch, bias will be added while calculating the batches.

outcomeMatrix - the data that denotes the outcome of the features- while supervised learning. Can be null for unsupervised methods.

batchSize - the batch size to use, 0 denotes full batch learning by default.

numThreads - the number of threads to use to calculate the batches.

stochastic - if true the batches will be evaluated stochastically, that means that each iteration the next batch is chosen to evaluate and calculate gradient and cost.

Method Detail

evaluateCost

public final CostGradientTuple evaluateCost(de.jungblut.math.DoubleVector input)

Description copied from interface: CostFunction

Evaluation for the cost function to retrieve cost and gradient.

Specified by:

evaluateCost in interface CostFunction

Parameters:

input - a given input vector

Returns:

a tuple consists of J (cost) and a vector X which is the gradient of the input.

evaluateBatch

protected abstract CostGradientTuple evaluateBatch(de.jungblut.math.DoubleVector theta,
                                                   de.jungblut.math.DoubleMatrix featureBatch,
                                                   de.jungblut.math.DoubleMatrix outcomeBatch)

Evaluate the batch.

Parameters:

theta - the parameters to use.

featureBatch - the batch matrix as input (already contains a bias!).

outcomeBatch - the batch matrix denoting the output.

Returns:

the cost/gradient tuple usually used when using evaluateCost(DoubleVector).