Assume that we have a positive definite matrix $C$, and a positive definite diagonal matrix $\Lambda$.

Are all the diagonal entries of $(C + \Lambda)^{-1}$ smaller than those of $C^{-1}$? In other words, if we increase the diagonal entries of a positive definite matrix, will the diagonal entries of its inverse decrease? If so, how to prove it?

Thanks a lot


The proof is as follows:

Consider $C^{-1}-(C+\Lambda)^{-1} $ $$ C^{-1}-(C+\Lambda)^{-1} = C^{-1} \left( [C+\Lambda] - C\right) (C+\Lambda)^{-1} =C^{-1} \Lambda (C+\Lambda)^{-1} = C^{-1} \Lambda (I+C^{-1}\Lambda)^{-1} C^{-1}= C^{-1} (\Lambda^{-1}+C^{-1})^{-1} C^{-1} $$ The last matrix is clearly positive definite. Hence, its diagonals must be positive. This shows that $$\left[C^{-1}\right]_{ii} >\left[(C+\Lambda)^{-1}\right]_{ii}$$

Added in response to OP's question

If $\Lambda$ is only positive semi definite, then $\Lambda + \epsilon I$ is positive definite for any $\epsilon >0$. Hence, by virtue of previous argument $$\left[C^{-1}\right]_{ii} >\left[(C+\Lambda+\epsilon I)^{-1}\right]_{ii}, \forall \epsilon > 0$$ By continuity argument, as $\epsilon \to 0$ $$\left[C^{-1}\right]_{ii} \ge \left[(C+\Lambda)^{-1}\right]_{ii}$$

Note that $\Lambda + \epsilon I$ is positive definite for any $\epsilon >0$ because $$ x^T (\Lambda + \epsilon I) x = x^T \Lambda x + \epsilon x^T x \ge \epsilon x^T x \gt 0 \text{ if $x \neq 0$}$$

The trick of adding $\epsilon I$ to a semi-definite matrix to make is positive definite and then taking the limit as $\epsilon \to 0^+$ is the standard way to extend results from definite to semidefinite.

  • $\begingroup$ I just come up with the same way to prove it. $\endgroup$ – Charles Zheng Jan 18 '14 at 5:45
  • $\begingroup$ what if $\Lambda$ is positive semidifinite, then $\Lambda$ can be non-invertible. $\endgroup$ – Charles Zheng Jan 18 '14 at 6:37
  • $\begingroup$ If $\Lambda$ is only positive semi-definite, then you use continuity argument to show that the result is true except you have to replace $\gt$ by $\ge$. The argument is to replace $\Lambda$ by $\Lambda + \epsilon I$ and take the limit as $\epsilon \to 0^+$ $\endgroup$ – user44197 Jan 18 '14 at 6:47
  • $\begingroup$ Why can we replace $\Lambda$ by $\Lambda + \epsilon I$? Besides, if $\Lambda$ is non-invertible, then the last step is not right. $\endgroup$ – Charles Zheng Jan 18 '14 at 6:54
  • $\begingroup$ If $\Lambda \ge 0$ then $\Lambda + \epsilon I \gt 0$ for all $\epsilon >0$ and hence is invertible. $\endgroup$ – user44197 Jan 18 '14 at 6:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.