de.jungblut.math.minimize

Class GradientDescent

java.lang.Object

de.jungblut.math.minimize.AbstractMinimizer

de.jungblut.math.minimize.GradientDescent

All Implemented Interfaces:

Minimizer

public final class GradientDescent
extends AbstractMinimizer

Gradient descent implementation with some neat features like momentum, divergence detection, delta breaks and bold driver and scheduled annealing adaptive learning rates. For more sophisticated configuration use the GradientDescent.GradientDescentBuilder.

Author:

thomas.jungblut

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	GradientDescent.GradientDescentBuilder

Constructor Summary

Constructors

Constructor and Description
GradientDescent(double alpha, double limit)

Method Summary

Modifier and Type	Method and Description
de.jungblut.math.DoubleVector	minimize(CostFunction f, de.jungblut.math.DoubleVector pInput, int maxIterations, boolean verbose) Minimizes the given costfunction with the starting parameter theta.
static de.jungblut.math.DoubleVector	minimizeFunction(CostFunction f, de.jungblut.math.DoubleVector pInput, double alpha, double limit, int length, boolean verbose) Minimize a given cost function f with the initial parameters pInput (also called theta) with a learning rate alpha and a fixed number of iterations.

Methods inherited from class de.jungblut.math.minimize.AbstractMinimizer

addIterationCompletionCallback, onIterationFinished

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

GradientDescent

public GradientDescent(double alpha,
                       double limit)

Parameters:

alpha - the learning rate.

limit - the delta in cost to archieve to break the iterations.

Method Detail

minimize

public final de.jungblut.math.DoubleVector minimize(CostFunction f,
                                                    de.jungblut.math.DoubleVector pInput,
                                                    int maxIterations,
                                                    boolean verbose)

Description copied from interface: Minimizer

Minimizes the given costfunction with the starting parameter theta.

Parameters:

f - the costfunction to minimize.

pInput - the starting parameters.

maxIterations - the number of iterations to do.

verbose - if TRUE it will print progress.

Returns:

the optimized theta parameters.

minimizeFunction

public static de.jungblut.math.DoubleVector minimizeFunction(CostFunction f,
                                                             de.jungblut.math.DoubleVector pInput,
                                                             double alpha,
                                                             double limit,
                                                             int length,
                                                             boolean verbose)

Minimize a given cost function f with the initial parameters pInput (also called theta) with a learning rate alpha and a fixed number of iterations. The loop can break earlier if costs converge below the limit. If the same cost was archieved three times in a row, it will also break the iterations.

Parameters:

f - the function to minimize.

pInput - the starting parameters.

alpha - the learning rate.

limit - the cost to archieve to break the iterations.

length - the number of iterations.

verbose - if true prints progress to STDOUT.

Returns:

the learned minimal parameters.