# Whether or not is $J^TJ+\lambda \operatorname{diag}(J^TJ)$ guaranteed to be invertible?

This problem comes from solving nonlinear least squares with sparse Levenberg-Marquardt algorithm, to derive a iteration increment $\Delta$ we have to solve following system: $$(J^TJ+\lambda \operatorname{diag}(J^TJ))\Delta=-J^T\epsilon$$ where $\operatorname{diag}(J^TJ)=\operatorname{diag}(j_1,j_2,j_3,....)$ is a diagonal matrix with diagonal elements corresponding to those of $J^TJ$ in turn, $\epsilon$ is the optimization object to be minimized.

Now write $x^T(J^TJ+\lambda \operatorname{diag}(J^TJ))x$ as $\|Jx\|^2+\sum_ij_ix_i^2$, it seems not unnecessarily positive even if $j_i \ge0$ for symmetric positive semi-definite matrix $J^TJ$

You are right, the matrix $M=J^TJ+\lambda\, \mathrm{diag}(J^TJ)$ on the left side of your displayed equation need not be invertible: if, as for instance when $J$ is the zero matrix or more generally when one of the $j_i = 0$. But you really want to solve the equation $M\Delta=-J^T\epsilon$, which can be solved, since the range of $M$ is the column space of $J^T$.
Suppose $j_1=0$, for example. Then the first entry in the right hand $-J^T\epsilon$ will equal $0$, as will all the entries in the first row and column of $M$. That it, the coefficient of $\Delta_1$ in your equation system is $0$, and you can reduce your equation set by dropping that variable. And so on.
• Cool! So $j_1=0$ can derive that first column of jacobian is a zero vector which leads to the first component of variable cannot affect the current iteration(i.e. object funtction is independent of the first component), so we can leave out increment at the first component. – Finley Jan 31 '18 at 14:58