Partial Ordering of Positive Definite Matrices Assume that $\mathbf{A}$, $\mathbf{B}$, and $\mathbf{A-B}$ are all positive definite Hermitian symmetric matrices of same dimensions. Prove that $\mathbf{B}^{-1}-\mathbf{A}^{-1}$ is positive definite.
 A: We use the following useful fact:

Let $\mathbf{A}$ and $\mathbf{B}$ be HPD (Hermitian positive definite). Then 
  $$\mathbf{A}\geq \mathbf{B} \quad \Leftrightarrow \quad \mathbf{I}\geq \mathbf{A}^{-1/2}\mathbf{B}\mathbf{A}^{-1/2}.$$

(Here $\mathbf{A}\geq \mathbf{B}$ means that $\mathbf{A}-\mathbf{B}$ is positive semidefinite and $\mathbf{I}$ is the identity matrix.)
Proof: $\mathbf{A}\geq \mathbf{B}$ if (and only if) for any $\mathbf{x}$ we have $\mathbf{x}^*\mathbf{A}\mathbf{x}\geq \mathbf{x}^*\mathbf{B}\mathbf{x}$. Since $\mathbf{A}$ is HPD, any $\mathbf{x}$ can be written as $\mathbf{x}=\mathbf{A}^{-1/2}\mathbf{y}$ and hence the previous inequality is equivalent to $$\mathbf{y}^*\mathbf{A}^{-1/2}\mathbf{A}\mathbf{A}^{-1/2}\mathbf{y}=\mathbf{y}^*\mathbf{I}\mathbf{y}\geq \mathbf{y}^*\mathbf{A}^{-1/2}\mathbf{B}\mathbf{A}^{-1/2}\mathbf{y}.$$ This means that $\mathbf{I}\geq \mathbf{A}^{-1/2}\mathbf{B}\mathbf{A}^{-1/2}$. Q.E.D.
Now we use twice the useful fact and the similarity relations to prove the statement of the question.
Let $\mathbf{A}$ and $\mathbf{B}$ be HPD and $\mathbf{A}\geq \mathbf{B}$. Using the useful fact we have that $\mathbf{I}-\mathbf{A}^{-1/2}\mathbf{B}\mathbf{A}^{-1/2}\geq 0$. We have the following similarity relations:
$$\mathbf{I}-\mathbf{A}^{-1/2}\mathbf{B}\mathbf{A}^{-1/2}\sim \mathbf{I}-\mathbf{A}^{-1}\mathbf{B}\sim \mathbf{I}-\mathbf{B}^{1/2}\mathbf{A}^{-1}\mathbf{B}^{1/2}.$$
Hence the matrix $\mathbf{I}-\mathbf{B}^{1/2}\mathbf{A}^{-1}\mathbf{B}^{1/2}$ is semidefinite. Using again the useful fact we have that $\mathbf{B}^{-1}\geq \mathbf{A}^{-1}$, that is, $\mathbf{B}^{-1}-\mathbf{A}^{-1}\geq 0$.
You can adapt it to your special case by replacing all occurrences of "$\geq$" and "semidefinite" by, respectively, "$>$" and "definite"; except in the proof of the useful fact where "any $\mathbf{x}$" must be replaced by "any nonzero $\mathbf{x}$".
A: First, suppose that $\|A-B\|<(\|A\|^{-1})^{-1}$. Writing $C=A-B$ we find that 
$$B^{-1} = (A-C)^{-1}=((I-CA^{-1})A)^{-1} = A^{-1}(I-CA^{-1})^{-1}$$
which expands into Neumann series
$$B^{-1} =  \sum_{n=0}^\infty A^{-1}(CA^{-1})^{n}$$
Hence, 
$$B^{-1}-A^{-1} =  \sum_{n=1}^\infty A^{-1}(CA^{-1})^{n}$$
where every term in the series, e.g., $A^{-1}CA^{-1}CA^{-1}$, is positive, being a symmetric word in  two positive matrices. 
The general case reduces to the above by considering the finite sequence $$A_k= A-\frac{k}{n}(A-B), \quad k=0,\dots,n$$ where $n$ is sufficiently large so that $$\frac{1}{n}\|A-B\|<\|B^{-1}\|^{-1}$$
holds. Apply the above to $A_{k-1}$ and $A_{k }$, $k=1,\dots,n$. (This is possible because the smallest eigenvalue of $A_{k-1}$ is bounded from below by the smallest eigenvalue of $B$).
