Positive definiteness of difference of inverse matrices

Let $A$ and $B$ be two $n \times n$ symmetric and positive definite matrices.

If $A \prec B$, then is it true that $B^{-1} \prec A^{-1}$?

Here, $A \prec B$ means that $B-A$ is positive definite.

For ease of notation I'll use $$\ge$$ instead of $$\succeq$$ and so on.

Lemma 1: $$A \le B \Rightarrow C^T A C \le C^T B C$$ for any conformable matrix $$C.$$

Proof. We have $$x^TC^T(B-A)C x = (Cx)^T(B-A)C x \ge 0$$ for any conformable vector $$x$$ so that $$C^T(B-A)C \ge 0.$$

Lemma 2: $$I \preceq B \Rightarrow$$ $$B$$ is invertible and $$B^{-1} \preceq I.$$

Proof.

• Since $$B$$ is symmetric we can find an orthogonal matrix $$G$$ and a diagonal matrix $$D$$ such that $$B=GDG^T.$$ Hence $$I = G^T I G \le G^T B G = D.$$ Thus all eigenvalues of $$B$$ are $$\ge 1 >0,$$ hence $$B$$ is invertible.
• Now write $$B^{-1} = B^{-1/2}IB^{-1/2} \le B^{-1/2}BB^{-1/2} = I,$$ where $$B^\alpha := GD^{\alpha}G^T.$$

Proposition: $$0 < A \le B \Rightarrow$$ $$B$$ invertible and $$B^{-1} \le A^{-1}.$$

Proof. \begin{align*} A\le B &\Rightarrow B-A \ge 0 \\ &\Rightarrow A^{-1/2}(B-A)A^{-1/2} \ge 0\\ &\Rightarrow A^{-1/2}BA^{-1/2} \ge I\\ &\Rightarrow A^{1/2}B^{-1}A^{1/2} \le I,\\ \end{align*} hence \begin{align*} B^{-1} &= A^{-1/2}A^{1/2}B^{-1}A^{1/2}A^{-1/2}\\ &\le A^{-1/2}IA^{-1/2}\\ &= A^{-1}. \end{align*}

• Should the last steps be simply $B^{-1}=A^{-1/2}(A^{1/2}B^{-1}A^{1/2})A^{-1/2}\leq A^{-1/2}IA^{-1/2}=A^{-1}$? Commented Nov 18, 2020 at 21:44
• @Knightgu yes that's correct, thanks for noticing! Commented Nov 20, 2020 at 9:37
• That's amazing. Thanks a lot for this very nice answer @Epiousios.
– user904326
Commented Mar 26, 2021 at 7:14

HINT:

Try proving that if $A$ is positive definite and $B-A$ is non-negative definite, then $\det(B-\lambda A)=0$ has all its roots $\lambda\ge 1$ and conversely if $\lambda\ge1$, then $B-A$ is non-negative definite. (it's an exercise from a book by the way)

Then, $\det(A^{-1}-\mu B^{-1})=0$

$\Rightarrow \det B\det(A^{-1}-\mu B^{-1})\det A\ge0$ $\quad($ as $A$ and $B$ are both p.d., $\det A,\det B>0)$

$\Rightarrow \det(B-\mu A)=0$

So the roots of $\det(A^{-1}-\mu B^{-1})=0$ are the same as the roots of $\det(B-\mu A)=0$.

As $B-A$ is n.n.d. we have $\mu\ge1$, which in turn makes $A^{-1}-B^{-1}$ n.n.d. (by the claim above)

• exercise from what book?
– a06e
Commented Jun 4, 2018 at 15:56
• @becko Linear Statistical Inference and it's Applications by C.R. Rao (not a linear algebra book). Commented Jun 4, 2018 at 16:03
• I had asked a question on the proof of the claim made here: math.stackexchange.com/q/2671111/321264 Commented Sep 15, 2018 at 7:01