If $A$ and $B$ are two positive definite matrices such that $A - B$ is nonnegative definite, is it true that $B^{-1} - A^{-1}$ is positive definite?
The doubt came to me when working with confidence regions in multivariate statistics that are usually obtained as hyper-ellipsoids.
Let $x \in \mathbb R^n$, let $y = A^{-1/2}x$. As $0 < B \le A$, we have $$ \def\<#1>{\left<#1\right>} 0 < \<A^{-1/2}BA^{-1/2}x,x> = \<By, y> \le \<Ay,y> = \<x,x> $$ That is $0 < A^{-1/2}BA^{-1/2} \le {\rm Id}$. Now note that $A^{-1/2}BA^{-1/2}$ is simlar to $BA^{-1}$, so $BA^{-1}$ has all its eigenvalues in $(0,1]$. Hence, its inverse $AB^{-1}$, has all its eigenvalues in $[1,\infty)$, so has the similar matrix $A^{1/2}B^{-1}A^{1/2}$, as this matrix is hermitian, we have ${\rm Id} \le A^{1/2}B^{-1}A^{1/2}$. This gives, arguing as the first step above, $A^{-1} \le B^{-1}$.
