I am minimizing a sum-of-squared-differences function using the Levenberg-Marquardt method. The off-the-shelf numerical implementations I have have looked through (MATLAB, Numerical Recipes in C (15.5), this implementation) all ask for a definition of the Jacobian. However, the paper that I am implementing gives only derivations for the gradient vector and Hessian matrix (pg. 4).

While the paper might give more clarity to my question, the short of it is that the function is an error function of the form:

$$\frac{1}{n}\sum_{i=1}^{n}[I(x,y,t) - I(x', y', t+1)]^2 = \frac{1}{n}\sum_{i=1}^{n}e_i^2 $$

where $I(x,y,t)$ is known and $I(x',y',t+1)$ is a function of some vector $\mathbf{f}$.

The objective is thus:

$$argmin_{\mathbf{f}} \left( \frac{1}{n}\sum_{i=1}^{n}e(\mathbf{f})_i^2 \right)$$

The paper says it to use Levenberg-Marquardt and provides:

...the approximate Hessian H and the gradient vector G can be computed,

$$\mathbf{h_{m,n}} = \sum_{k} \frac{\partial e_k}{\partial f_i}\frac{\partial e_k}{\partial f_j}, \mathbf{g_i} = \sum_{k}\frac{\partial e_k}{\partial f_i} $$

All of this brings me to my question:

Why might I need this Hessian approximation and gradient for Levenberg-Marquardt? In general doesn't L-M just need the Jacobian which is the essentially this gradient without summing each column $i$ (ie. $\mathbf{J_{k,i}} = \frac{\partial e_k}{\partial f_i}$)?

  • $\begingroup$ What is it that you're trying to compute the Jacobian of? What is it that you have the gradient/Hessian of? $\endgroup$ Dec 19 '15 at 18:54
  • $\begingroup$ @Omnomnomnom: It's a sum of squared differences function (I just edited OP to include this) $\endgroup$
    – marcman
    Dec 19 '15 at 18:55
  • $\begingroup$ I think the confusion arises from the difference between solving a nonlinear system (by least squares) and optimizing a function. The former has a Jacobian (partials of the various functions you seek a simultaneous root of with respect to the arguments) and the latter has a Hessian (matrix of second partials, though this might be approximated). $\endgroup$
    – hardmath
    Dec 21 '15 at 20:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.