de.jungblut.classification.nn

Class MultilayerPerceptronCostFunction

java.lang.Object

de.jungblut.math.minimize.AbstractMiniBatchCostFunction

de.jungblut.classification.nn.MultilayerPerceptronCostFunction

All Implemented Interfaces:

CostFunction

public final class MultilayerPerceptronCostFunction
extends AbstractMiniBatchCostFunction

Neural network costfunction for a multilayer perceptron.

Author:

thomas.jungblut

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	MultilayerPerceptronCostFunction.NetworkConfiguration

Constructor Summary

Constructors

Constructor and Description
MultilayerPerceptronCostFunction(MultilayerPerceptron network, de.jungblut.math.DoubleVector[] features, de.jungblut.math.DoubleVector[] outcome)

Method Summary

Modifier and Type	Method and Description
static de.jungblut.math.DoubleMatrix[]	backwardPropagate(de.jungblut.math.DoubleMatrix y, de.jungblut.math.DoubleMatrix[] thetas, de.jungblut.math.DoubleMatrix[] ax, de.jungblut.math.DoubleMatrix[] zx, MultilayerPerceptronCostFunction.NetworkConfiguration conf)
static void	calculateGradients(de.jungblut.math.DoubleMatrix[] thetas, de.jungblut.math.DoubleMatrix[] thetaGradients, de.jungblut.math.DoubleMatrix[] ax, de.jungblut.math.DoubleMatrix[] deltaX, int m, MultilayerPerceptronCostFunction.NetworkConfiguration conf)
static double	calculateRegularization(de.jungblut.math.DoubleMatrix[] thetas, int m, MultilayerPerceptronCostFunction.NetworkConfiguration conf)
static CostGradientTuple	computeNextStep(de.jungblut.math.DoubleVector input, de.jungblut.math.DoubleMatrix x, de.jungblut.math.DoubleMatrix y, MultilayerPerceptronCostFunction.NetworkConfiguration conf) Do a full forward pass and backpropagate the error.
static int[][]	computeUnfoldParameters(int[] layerSizes) Calculates the unfold parameters to unroll a learned theta vector in their matrix.
static void	dropout(Random rnd, de.jungblut.math.DoubleMatrix activations, double p) Computes dropout for the activations matrix.
static void	dropoutVisibleLayer(de.jungblut.math.DoubleMatrix x, de.jungblut.math.DoubleMatrix[] ax, MultilayerPerceptronCostFunction.NetworkConfiguration conf)
protected CostGradientTuple	evaluateBatch(de.jungblut.math.DoubleVector theta, de.jungblut.math.DoubleMatrix featureBatch, de.jungblut.math.DoubleMatrix outcomeBatch) Evaluate the batch.
static void	forwardPropagate(de.jungblut.math.DoubleMatrix[] thetas, de.jungblut.math.DoubleMatrix[] ax, de.jungblut.math.DoubleMatrix[] zx, MultilayerPerceptronCostFunction.NetworkConfiguration conf)

Methods inherited from class de.jungblut.math.minimize.AbstractMiniBatchCostFunction

evaluateCost

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

MultilayerPerceptronCostFunction

public MultilayerPerceptronCostFunction(MultilayerPerceptron network,
                                        de.jungblut.math.DoubleVector[] features,
                                        de.jungblut.math.DoubleVector[] outcome)

Method Detail

evaluateBatch

protected CostGradientTuple evaluateBatch(de.jungblut.math.DoubleVector theta,
                                          de.jungblut.math.DoubleMatrix featureBatch,
                                          de.jungblut.math.DoubleMatrix outcomeBatch)

Description copied from class: AbstractMiniBatchCostFunction

Evaluate the batch.

Specified by:

evaluateBatch in class AbstractMiniBatchCostFunction

Parameters:

theta - the parameters to use.

featureBatch - the batch matrix as input (already contains a bias!).

outcomeBatch - the batch matrix denoting the output.

Returns:

the cost/gradient tuple usually used when using AbstractMiniBatchCostFunction.evaluateCost(DoubleVector).

computeNextStep

public static CostGradientTuple computeNextStep(de.jungblut.math.DoubleVector input,
                                                de.jungblut.math.DoubleMatrix x,
                                                de.jungblut.math.DoubleMatrix y,
                                                MultilayerPerceptronCostFunction.NetworkConfiguration conf)

Do a full forward pass and backpropagate the error.

Parameters:

input - the input parameters (theta).

x - the features.

y - the outcome.

Returns:

a tuple of cost and gradient.

forwardPropagate

public static void forwardPropagate(de.jungblut.math.DoubleMatrix[] thetas,
                                    de.jungblut.math.DoubleMatrix[] ax,
                                    de.jungblut.math.DoubleMatrix[] zx,
                                    MultilayerPerceptronCostFunction.NetworkConfiguration conf)

backwardPropagate

public static de.jungblut.math.DoubleMatrix[] backwardPropagate(de.jungblut.math.DoubleMatrix y,
                                                                de.jungblut.math.DoubleMatrix[] thetas,
                                                                de.jungblut.math.DoubleMatrix[] ax,
                                                                de.jungblut.math.DoubleMatrix[] zx,
                                                                MultilayerPerceptronCostFunction.NetworkConfiguration conf)

calculateGradients

public static void calculateGradients(de.jungblut.math.DoubleMatrix[] thetas,
                                      de.jungblut.math.DoubleMatrix[] thetaGradients,
                                      de.jungblut.math.DoubleMatrix[] ax,
                                      de.jungblut.math.DoubleMatrix[] deltaX,
                                      int m,
                                      MultilayerPerceptronCostFunction.NetworkConfiguration conf)

calculateRegularization

public static double calculateRegularization(de.jungblut.math.DoubleMatrix[] thetas,
                                             int m,
                                             MultilayerPerceptronCostFunction.NetworkConfiguration conf)

dropoutVisibleLayer

public static void dropoutVisibleLayer(de.jungblut.math.DoubleMatrix x,
                                       de.jungblut.math.DoubleMatrix[] ax,
                                       MultilayerPerceptronCostFunction.NetworkConfiguration conf)

computeUnfoldParameters

public static int[][] computeUnfoldParameters(int[] layerSizes)

Calculates the unfold parameters to unroll a learned theta vector in their matrix.

Parameters:

layerSizes - the layer size that the MultilayerPerceptron got instantiated with.

Returns:

the unfold parameters to feed DenseMatrixFolder with.

dropout

public static void dropout(Random rnd,
                           de.jungblut.math.DoubleMatrix activations,
                           double p)

Computes dropout for the activations matrix. Each element for each row has the similar probability p to be "dropped out" (set to 0) of the computation. This way, the network does learn to not rely on other units thus learning to detect more general features than drastically overfitting the dataset.

Parameters:

rnd - the random number generator to consult.

activations - activations of units per record on each column.

p - dropout probability.