# How do I construct the Jacobian for use in a Levenberg-Marquardt algorithm.

I am working on a 3D reconstruction system and I am looking to use a Levenberg-marquardt algorithm to do bundle adjustment. I am not too sure about how LM works and what it requires.

The model I am working with has multiple equations each with a number of parameters that are unknown and have to be found. The equations share a number of unknown parameters between them. I.e.

$$y_1 = u^2 + v^2$$

$$y_2 = u^2 + v^2 + k^2$$

where u, v and k are unknowns.

Am I correct in assuming that the first thing I should do is work out the Jacobian of the set of equations? Once I work out the Jacobian can I run that through the LM algorithm with the observed results to find out my parameters?

The equations work out a value that is a 2D coordinate. Do I have to split up the equations into $2$ separate $x$ and $y$ components and add them both as separate equations into the jacobian? or can I use the $(x,y)$ coordinate in a single equation?

Finally I realise that will be a lot of parameters that are not shared a lot of the equations, meaning that a lot of the first order derivatives will be $0$ values. Does that matter for LM?

Thanks in advance, hopefully this all makes sense.

-
"a value that is a 2D coordinate" - your unknowns are all vectors? – J. M. Feb 2 '12 at 17:35
The unknowns are all rotation values in radians or 3D vectors. The end result of plugging in all the unknowns into the model will generate a 2D coordinate representing a pixel location on a screen. – Jkh2 Feb 2 '12 at 18:44
I think, you should first work out how your quadratic cost ("chi2 error", "negative loglikelihood", "energy", or how you call it) exactly looks like. A typical Bundle Adjustment cost term looks like this: $\sum_{\mathbf{z}_{i,j} } (\mathbf{z}_{i,j} - \mathbf{\hat{z}}(\mathtt{T_i}, \mathbf{x_j}))^2$, where $\mathtt{T_i}$ is the $i$th 6d pose, $\mathbf{x_j}$ the $j$th 3d point, and $\mathbf{z}_i$ is a 2d measurement of the $j$th point from the $i$th pose. And $\mathbf{\hat{z}}$ is the projection function/forward model. What are $u$, $v$ and $k$? Are you trying to estimate intrinsics? – Hauke Strasdat Feb 3 '12 at 2:25
I was using u, v and k as examples. The values I am trying to work out are rotation values used in the forward model. – Jkh2 Feb 5 '12 at 15:54
I am trying to minimise the distance between the observed 2D point and the 2D point obtained using the forward model. I now believe I have to work out the jacobian of my equations that are used for the forward model. I am guessing I have to break each equation into x and y parts and differentiate each part with respect to each unknown parameter in my parameter vector, am I on the right track? – Jkh2 Feb 5 '12 at 16:05