LevenbergMarquardt (finmath lib 4.0.11 API)

java.lang.Object
- net.finmath.optimizer.LevenbergMarquardt

All Implemented Interfaces:

Serializable, Cloneable, Optimizer
```
public abstract class LevenbergMarquardt
extends Object
implements Serializable, Cloneable, Optimizer
```
This class implements a parallel Levenberg-Marquardt non-linear least-squares fit algorithm.
The solver minimizes \( || f ||_{L_{2}} \) for a function \( f:\mathbb{R}^n \rightarrow \mathbb{R}^m \). The solver requires the calculation of a Jacobi-matrix \( J = \frac{\mathrm{d}f}{\mathrm{d}x} \). The iteration steps are then defined by \[ \Delta x = H_{\lambda}^{-1} J^T f \] where \( H_{\lambda} \) is a regularized approximation of the Hessian matrix. The solver supports two different regularizations. For RegularizationMethod.LEVENBERG the solver uses \( H_{\lambda} = J^T J + \lambda I \). For RegularizationMethod.LEVENBERG_MARQUARDT the solver uses \( H_{\lambda} = J^T J + \lambda \text{diag}(J^T J) \).

The design avoids the need to define the objective function as a separate class. The objective function is defined by overriding a class method, see the sample code below.

The Levenberg-Marquardt solver is implemented in using multi-threading. The calculation of the derivatives (in case a specific implementation of setDerivatives(double[] parameters, double[][] derivatives) is not provided) may be performed in parallel by setting the parameter numberOfThreads.

To use the solver inherit from it and implement the objective function as setValues(double[] parameters, double[] values) where values has to be set to the value of the objective functions for the given parameters.
You may also provide an a derivative for your objective function by additionally overriding the function setDerivatives(double[] parameters, double[][] derivatives), otherwise the solver will calculate the derivative via finite differences.

To reject a point, it is allowed to set an element of values to Double.NaN in the implementation of setValues(double[] parameters, double[] values). Put differently: The solver handles NaN values in values as an error larger than the current one (regardless of the current error) and rejects the point.
Note, however, that is is an error if the initial parameter guess results in an NaN value. That is, the solver should be initialized with an initial parameter in an admissible region.
The following simple example finds a solution for the equation

Sample linear system of equations.
0.0 * x₁ + 1.0 * x₂ = 5.0

2.0 * x₁ + 1.0 * x₂ = 10.0
```
 
        LevenbergMarquardt optimizer = new LevenbergMarquardt() {
                // Override your objective function here
                public void setValues(double[] parameters, double[] values) {
                        values[0] = parameters[0] * 0.0 + parameters[1];
                        values[1] = parameters[0] * 2.0 + parameters[1];
                }
        };

        // Set solver parameters
        optimizer.setInitialParameters(new double[] { 0, 0 });
        optimizer.setWeights(new double[] { 1, 1 });
        optimizer.setMaxIteration(100);
        optimizer.setTargetValues(new double[] { 5, 10 });

        optimizer.run();

        double[] bestParameters = optimizer.getBestFitParameters();
 
 
```
See the example in the main method below.
The class can be initialized to use a multi-threaded valuation. If initialized this way the implementation of setValues must be thread-safe. The solver will evaluate the gradient of the value vector in parallel, i.e., use as many threads as the number of parameters.
Note: Iteration steps will be logged (java.util.logging) with LogLevel.FINE
Version:

1.6

Author:

Christian Fries

See Also:

Serialized Form

Sample linear system of equations.
0.0 * x₁ + 1.0 * x₂ = 5.0
2.0 * x₁ + 1.0 * x₂ = 10.0

Nested Class Summary

Nested Classes
Modifier and Type	Class and Description
`static class`	`LevenbergMarquardt.RegularizationMethod` The regularization method used to invert the approximation of the Hessian matrix.

Nested classes/interfaces inherited from interface net.finmath.optimizer.Optimizer
Optimizer.ObjectiveFunction

Constructor Summary

Constructors
Constructor and Description
`LevenbergMarquardt()` Create a Levenberg-Marquardt solver.
`LevenbergMarquardt(double[] initialParameters, double[] targetValues, int maxIteration, ExecutorService executorService)` Create a Levenberg-Marquardt solver.
`LevenbergMarquardt(double[] initialParameters, double[] targetValues, int maxIteration, int numberOfThreads)` Create a Levenberg-Marquardt solver.
`LevenbergMarquardt(int numberOfThreads)` Create a Levenberg-Marquardt solver.
`LevenbergMarquardt(LevenbergMarquardt.RegularizationMethod regularizationMethod, double[] initialParameters, double[] targetValues, int maxIteration, ExecutorService executorService)` Create a Levenberg-Marquardt solver.
`LevenbergMarquardt(LevenbergMarquardt.RegularizationMethod regularizationMethod, double[] initialParameters, double[] targetValues, int maxIteration, int numberOfThreads)` Create a Levenberg-Marquardt solver.
`LevenbergMarquardt(List<Number> initialParameters, List<Number> targetValues, int maxIteration, ExecutorService executorService)` Create a Levenberg-Marquardt solver.
`LevenbergMarquardt(List<Number> initialParameters, List<Number> targetValues, int maxIteration, int numberOfThreads)` Create a Levenberg-Marquardt solver.

Method Summary

All Methods Static Methods Instance Methods Abstract Methods Concrete Methods
Modifier and Type	Method and Description
`LevenbergMarquardt`	`clone()` Create a clone of this LevenbergMarquardt optimizer.
`double[]`	`getBestFitParameters()` Get the best fit parameter vector.
`LevenbergMarquardt`	`getCloneWithModifiedTargetValues(double[] newTargetVaues, double[] newWeights, boolean isUseBestParametersAsInitialParameters)` Create a clone of this LevenbergMarquardt optimizer with a new vector for the target values and weights.
`LevenbergMarquardt`	`getCloneWithModifiedTargetValues(List<Number> newTargetVaues, List<Number> newWeights, boolean isUseBestParametersAsInitialParameters)` Create a clone of this LevenbergMarquardt optimizer with a new vector for the target values and weights.
`int`	`getIterations()` Get the number of iterations.
`double`	`getLambda()` Get the parameter λ used in the Tikhonov-like regularization of the Hessian matrix, that is the \( \lambda \) in \( H + \lambda \diag H \).
`double`	`getLambdaDivisor()` Get the divisor applied to lambda (for the next iteration) if the inversion of regularized Hessian succeeds, that is, if \( H + \lambda \diag H \) is invertable.
`double`	`getLambdaMultiplicator()` Get the multiplicator applied to lambda if the inversion of regularized Hessian fails, that is, if \( H + \lambda \diag H \) is not invertable.
`double`	`getMeanSquaredError(double[] value)`
`double`	`getRootMeanSquaredError()`
`static void`	`main(String[] args)`
`void`	`run()` Runs the optimization.
`void`	`setDerivatives(double[] parameters, double[][] derivatives)` The derivative of the objective function.
`LevenbergMarquardt`	`setErrorTolerance(double errorTolerance)` Set the error tolerance.
`LevenbergMarquardt`	`setInitialParameters(double[] initialParameters)` Set the initial parameters for the solver.
`void`	`setLambda(double lambda)` Set the parameter λ used in the Tikhonov-like regularization of the Hessian matrix, that is the \( \lambda \) in \( H + \lambda \diag H \).
`void`	`setLambdaDivisor(double lambdaDivisor)` Set the divisor applied to lambda (for the next iteration) if the inversion of regularized Hessian succeeds, that is, if \( H + \lambda \diag H \) is invertable.
`void`	`setLambdaMultiplicator(double lambdaMultiplicator)` Set the multiplicator applied to lambda if the inversion of regularized Hessian fails, that is, if \( H + \lambda \diag H \) is not invertable.
`LevenbergMarquardt`	`setMaxIteration(int maxIteration)` Set the maximum number of iterations to be performed until the solver gives up.
`LevenbergMarquardt`	`setParameterSteps(double[] parameterSteps)` Set the parameter step for the solver.
`LevenbergMarquardt`	`setTargetValues(double[] targetValues)` Set the target values for the solver.
`abstract void`	`setValues(double[] parameters, double[] values)` The objective function.
`LevenbergMarquardt`	`setWeights(double[] weights)` Set the weight for the objective function.

Methods inherited from class java.lang.Object
equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Constructor Detail
  - LevenbergMarquardt
```
public LevenbergMarquardt(LevenbergMarquardt.RegularizationMethod regularizationMethod,
                          double[] initialParameters,
                          double[] targetValues,
                          int maxIteration,
                          ExecutorService executorService)
```
    Create a Levenberg-Marquardt solver.
    
    Parameters:
    
    regularizationMethod - The regularization method to use. See LevenbergMarquardt.RegularizationMethod.
    
    initialParameters - Initial value for the parameters where the solver starts its search.
    
    targetValues - Target values to achieve.
    
    maxIteration - Maximum number of iterations.
    
    executorService - Executor to be used for concurrent valuation of the derivatives. This is only performed if setDerivative is not overwritten. Warning: The implementation of setValues has to be thread safe!
  - LevenbergMarquardt
```
public LevenbergMarquardt(double[] initialParameters,
                          double[] targetValues,
                          int maxIteration,
                          ExecutorService executorService)
```
    Create a Levenberg-Marquardt solver.
    
    Parameters:
    
    initialParameters - Initial value for the parameters where the solver starts its search.
    
    targetValues - Target values to achieve.
    
    maxIteration - Maximum number of iterations.
    
    executorService - Executor to be used for concurrent valuation of the derivatives. This is only performed if setDerivative is not overwritten. Warning: The implementation of setValues has to be thread safe!
  - LevenbergMarquardt
```
public LevenbergMarquardt(LevenbergMarquardt.RegularizationMethod regularizationMethod,
                          double[] initialParameters,
                          double[] targetValues,
                          int maxIteration,
                          int numberOfThreads)
```
    Create a Levenberg-Marquardt solver.
    
    Parameters:
    
    regularizationMethod - The regularization method to use. See LevenbergMarquardt.RegularizationMethod.
    
    initialParameters - Initial value for the parameters where the solver starts its search.
    
    targetValues - Target values to achieve.
    
    maxIteration - Maximum number of iterations.
    
    numberOfThreads - Maximum number of threads. Warning: If this number is larger than one, the implementation of setValues has to be thread safe!
  - LevenbergMarquardt
```
public LevenbergMarquardt(double[] initialParameters,
                          double[] targetValues,
                          int maxIteration,
                          int numberOfThreads)
```
    Create a Levenberg-Marquardt solver.
    
    Parameters:
    
    initialParameters - Initial value for the parameters where the solver starts its search.
    
    targetValues - Target values to achieve.
    
    maxIteration - Maximum number of iterations.
    
    numberOfThreads - Maximum number of threads. Warning: If this number is larger than one, the implementation of setValues has to be thread safe!
  - LevenbergMarquardt
```
public LevenbergMarquardt(List<Number> initialParameters,
                          List<Number> targetValues,
                          int maxIteration,
                          ExecutorService executorService)
```
    Create a Levenberg-Marquardt solver.
    
    Parameters:
    
    initialParameters - List of initial values for the parameters where the solver starts its search.
    
    targetValues - List of target values to achieve.
    
    maxIteration - Maximum number of iterations.
    
    executorService - Executor to be used for concurrent valuation of the derivatives. This is only performed if setDerivative is not overwritten. Warning: The implementation of setValues has to be thread safe!
  - LevenbergMarquardt
```
public LevenbergMarquardt(List<Number> initialParameters,
                          List<Number> targetValues,
                          int maxIteration,
                          int numberOfThreads)
```
    Create a Levenberg-Marquardt solver.
    
    Parameters:
    
    initialParameters - Initial value for the parameters where the solver starts its search.
    
    targetValues - Target values to achieve.
    
    maxIteration - Maximum number of iterations.
    
    numberOfThreads - Maximum number of threads. Warning: If this number is larger than one, the implementation of setValues has to be thread safe!
  - LevenbergMarquardt
```
public LevenbergMarquardt()
```
    Create a Levenberg-Marquardt solver.
  - LevenbergMarquardt
```
public LevenbergMarquardt(int numberOfThreads)
```
    Create a Levenberg-Marquardt solver.
    
    Parameters:
    
    numberOfThreads - Maximum number of threads. Warning: If this number is larger than one, the implementation of setValues has to be thread safe!
- Method Detail
  - main
```
public static void main(String[] args)
                 throws SolverException,
                        CloneNotSupportedException
```
    Throws:
    
    SolverException
    
    CloneNotSupportedException
  - setInitialParameters
```
public LevenbergMarquardt setInitialParameters(double[] initialParameters)
```
    Set the initial parameters for the solver.
    
    Parameters:
    
    initialParameters - The initial parameters.
    
    Returns:
    
    A self reference.
  - setParameterSteps
```
public LevenbergMarquardt setParameterSteps(double[] parameterSteps)
```
    Set the parameter step for the solver. The parameter step is used to evaluate the derivatives via finite differences, if analytic derivatives are not provided.
    
    Parameters:
    
    parameterSteps - The parameter step.
    
    Returns:
    
    A self reference.
  - setTargetValues
```
public LevenbergMarquardt setTargetValues(double[] targetValues)
```
    Set the target values for the solver. The solver will solver the equation weights * objectiveFunction = targetValues.
    
    Parameters:
    
    targetValues - The target values.
    
    Returns:
    
    A self reference.
  - setMaxIteration
```
public LevenbergMarquardt setMaxIteration(int maxIteration)
```
    Set the maximum number of iterations to be performed until the solver gives up.
    
    Parameters:
    
    maxIteration - The maximum number of iterations.
    
    Returns:
    
    A self reference.
  - setWeights
```
public LevenbergMarquardt setWeights(double[] weights)
```
    Set the weight for the objective function.
    
    Parameters:
    
    weights - The weights for the objective function.
    
    Returns:
    
    A self reference.
  - setErrorTolerance
```
public LevenbergMarquardt setErrorTolerance(double errorTolerance)
```
    Set the error tolerance. The solver considers the solution "found" if the error is not improving by this given error tolerance.
    
    Parameters:
    
    errorTolerance - The error tolerance.
    
    Returns:
    
    A self reference.
  - getLambda
```
public double getLambda()
```
    Get the parameter λ used in the Tikhonov-like regularization of the Hessian matrix, that is the \( \lambda \) in \( H + \lambda \diag H \).
    
    Returns:
    
    the parameter \( \lambda \).
  - setLambda
```
public void setLambda(double lambda)
```
    Set the parameter λ used in the Tikhonov-like regularization of the Hessian matrix, that is the \( \lambda \) in \( H + \lambda \diag H \).
    
    Parameters:
    
    lambda - the lambda to set
  - getLambdaMultiplicator
```
public double getLambdaMultiplicator()
```
    Get the multiplicator applied to lambda if the inversion of regularized Hessian fails, that is, if \( H + \lambda \diag H \) is not invertable.
    
    Returns:
    
    the lambdaMultiplicator
  - setLambdaMultiplicator
```
public void setLambdaMultiplicator(double lambdaMultiplicator)
```
    Set the multiplicator applied to lambda if the inversion of regularized Hessian fails, that is, if \( H + \lambda \diag H \) is not invertable. This will make lambda larger, hence let the stepping move slower.
    
    Parameters:
    
    lambdaMultiplicator - the lambdaMultiplicator to set. Should be > 1.
  - getLambdaDivisor
```
public double getLambdaDivisor()
```
    Get the divisor applied to lambda (for the next iteration) if the inversion of regularized Hessian succeeds, that is, if \( H + \lambda \diag H \) is invertable.
    
    Returns:
    
    the lambdaDivisor
  - setLambdaDivisor
```
public void setLambdaDivisor(double lambdaDivisor)
```
    Set the divisor applied to lambda (for the next iteration) if the inversion of regularized Hessian succeeds, that is, if \( H + \lambda \diag H \) is invertable. This will make lambda smaller, hence let the stepping move faster.
    
    Parameters:
    
    lambdaDivisor - the lambdaDivisor to set. Should be > 1.
  - getBestFitParameters
```
public double[] getBestFitParameters()
```
    Description copied from interface: Optimizer
    
    Get the best fit parameter vector.
    
    Specified by:
    
    getBestFitParameters in interface Optimizer
    
    Returns:
    
    The best fit parameter.
  - getRootMeanSquaredError
```
public double getRootMeanSquaredError()
```
    Specified by:
    
    getRootMeanSquaredError in interface Optimizer
    
    Returns:
    
    the the root mean square error of achieved with the the best fit parameter
  - getIterations
```
public int getIterations()
```
    Description copied from interface: Optimizer
    
    Get the number of iterations.
    
    Specified by:
    
    getIterations in interface Optimizer
    
    Returns:
    
    The number of iterations required
  - setValues
```
public abstract void setValues(double[] parameters,
                               double[] values)
                        throws SolverException
```
    The objective function. Override this method to implement your custom function.
    
    Parameters:
    
    parameters - Input value. The parameter vector.
    
    values - Output value. The vector of values f(i,parameters), i=1,...,n
    
    Throws:
    
    SolverException - Thrown if the valuation fails, specific cause may be available via the cause() method.
  - setDerivatives
```
public void setDerivatives(double[] parameters,
                           double[][] derivatives)
                    throws SolverException
```
    The derivative of the objective function. You may override this method if you like to implement your own derivative.
    
    Parameters:
    
    parameters - Input value. The parameter vector.
    
    derivatives - Output value, where derivatives[i][j] is d(value(j)) / d(parameters(i)
    
    Throws:
    
    SolverException - Thrown if the valuation fails, specific cause may be available via the cause() method.
  - run
```
public void run()
         throws SolverException
```
    Description copied from interface: Optimizer
    
    Runs the optimization.
    
    Specified by:
    
    run in interface Optimizer
    
    Throws:
    
    SolverException - Thrown if the valuation fails, specific cause may be available via the cause() method.
  - getMeanSquaredError
```
public double getMeanSquaredError(double[] value)
```
  - clone
```
public LevenbergMarquardt clone()
                         throws CloneNotSupportedException
```
    Create a clone of this LevenbergMarquardt optimizer. The clone will use the same objective function than this implementation, i.e., the implementation of setValues(double[], double[]) and that of setDerivatives(double[], double[][]) is reused.
    
    Overrides:
    
    clone in class Object
    
    Throws:
    
    CloneNotSupportedException
  - getCloneWithModifiedTargetValues
```
public LevenbergMarquardt getCloneWithModifiedTargetValues(double[] newTargetVaues,
                                                           double[] newWeights,
                                                           boolean isUseBestParametersAsInitialParameters)
                                                    throws CloneNotSupportedException
```
    Create a clone of this LevenbergMarquardt optimizer with a new vector for the target values and weights. The clone will use the same objective function than this implementation, i.e., the implementation of setValues(double[], double[]) and that of setDerivatives(double[], double[][]) is reused. The initial values of the cloned optimizer will either be the original initial values of this object or the best parameters obtained by this optimizer, the latter is used only if this optimized signals a done().
    
    Parameters:
    
    newTargetVaues - New array of target values.
    
    newWeights - New array of weights.
    
    isUseBestParametersAsInitialParameters - If true and this optimizer is done(), then the clone will use this.getBestFitParameters() as initial parameters.
    
    Returns:
    
    A new LevenbergMarquardt optimizer, cloning this one except modified target values and weights.
    
    Throws:
    
    CloneNotSupportedException - Thrown if this optimizer cannot be cloned.
  - getCloneWithModifiedTargetValues
```
public LevenbergMarquardt getCloneWithModifiedTargetValues(List<Number> newTargetVaues,
                                                           List<Number> newWeights,
                                                           boolean isUseBestParametersAsInitialParameters)
                                                    throws CloneNotSupportedException
```
    Create a clone of this LevenbergMarquardt optimizer with a new vector for the target values and weights. The clone will use the same objective function than this implementation, i.e., the implementation of setValues(double[], double[]) and that of setDerivatives(double[], double[][]) is reused. The initial values of the cloned optimizer will either be the original initial values of this object or the best parameters obtained by this optimizer, the latter is used only if this optimized signals a done().
    
    Parameters:
    
    newTargetVaues - New list of target values.
    
    newWeights - New list of weights.
    
    isUseBestParametersAsInitialParameters - If true and this optimizer is done(), then the clone will use this.getBestFitParameters() as initial parameters.
    
    Returns:
    
    A new LevenbergMarquardt optimizer, cloning this one except modified target values and weights.
    
    Throws:
    
    CloneNotSupportedException - Thrown if this optimizer cannot be cloned.

Class LevenbergMarquardt

Nested Class Summary

Nested classes/interfaces inherited from interface net.finmath.optimizer.Optimizer

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Constructor Detail

LevenbergMarquardt

LevenbergMarquardt

LevenbergMarquardt

LevenbergMarquardt

LevenbergMarquardt

LevenbergMarquardt

LevenbergMarquardt

LevenbergMarquardt

Method Detail

main

setInitialParameters

setParameterSteps

setTargetValues

setMaxIteration

setWeights

setErrorTolerance

getLambda

setLambda

getLambdaMultiplicator

setLambdaMultiplicator

getLambdaDivisor

setLambdaDivisor

getBestFitParameters

getRootMeanSquaredError

getIterations

setValues

setDerivatives

run

getMeanSquaredError

clone

getCloneWithModifiedTargetValues

getCloneWithModifiedTargetValues