Derivative of a map involving the matrix inverse

Question

I have $f: U\rightarrow \mathbb{R}$, $f(X):=\operatorname{tr}(X^{-1})$, $U$ contains all matrices $X$, which are positive definite and symmetric. I want to show that $f$ is differentiable on $U$.

To do so, I have to figure out $f(X+tY)=\operatorname{tr}[(X+tY)^{-1}]=\operatorname{tr}[ (X(I-(-t)X^{-1}Y))^{-1}]$ Now it is: $(X(I-(-t)X^{-1}Y))^{-1}=\sum_{i=0}^{\infty}(-t)^{i}(X^{-1}Y)^{i}X^{-1}$. This is the point where I get stuck. The first term of the series vanishes with $\operatorname{tr}[X]$, when it comes to the derivation. But how can I fix the rest?

Answer 1

$g:\begin{array}{ll}S_n^{++}\left(\Bbb R\right)\to S_n^{++}\left(\Bbb R\right)\\ X \mapsto X^{-1}\end{array}$

$h:\begin{array}{ll}S_n^{++}\left(\Bbb R\right) \to \Bbb R\\X\mapsto \operatorname{Tr}(X)\end{array}$

$f=h\circ g$

---

Let $H\in M_n\left(\Bbb R\right)$ so that $I_n+H\in S_n^{++}\left(\Bbb R\right)$ and $\left\|H\right\|< 1$

$g(I_n+H)=(I_n+H)^{-1}=\sum\limits_{k=0}^{+\infty}(-H)^k=I_n-H+\sum\limits_{k=2}^{+\infty}(-H)^k$

$\left\|g(I_n+H)-(I_n-H)\right\|=\left\|\sum\limits_{k=2}^{+\infty}(-H)^k\right\|\le \sum\limits_{k=2}^{+\infty}\left\|(-H)\right\|^k=\cfrac{\|H\|^2}{1-\left\|H\right\|}$

$\cfrac{\left\|g(I_n+H)-g(I_n)-(-H))\right\|}{\|H\|}=\cfrac{\|H\|}{1-\left\|H\right\|}\underset{\left\|H\right\|\to 0}{\longrightarrow}0$

So $g$ is differentiable at $I_n$ and $D_{I_n}(H)=-H$

---

Let $X\in S_n^{++}\left(\Bbb R\right)$

Let $H\in M_n\left(\Bbb R\right)$ so that $X+H\in S_n^{++}\left(\Bbb R\right)$ and $\left\|H\right\|< \cfrac{1}{\|X^{-1}\|}$ (so that $\|X^{-1}H\|\le \|X^{-1}\| \|H\| <1$). Note that I am using the operator norm to get $\|X^{-1}H\|\le \|X^{-1}\| \|H\|$ (and I can do that because the space is finite dimensional so all norms are equivalent).

$\begin{array}{ll}g(X+H)&=g(X(I_n+X^{-1}H))\\&=(X(I_n+X^{-1}H))^{-1}\\&=(I_n+X^{-1}H)^{-1}X^{-1}\\&=(I_n-X^{-1}H+o(H))X^{-1}\\&=X^{-1}-X^{-1}HX^{-1}+o(H)X^{-1}\end{array}$

$\cfrac{\|g(X+H)-g(X)-(-X^{-1}HX^{-1})\|}{\|H\|}=\cfrac{\|o(H)X^{-1}\|}{\|H\|}\underset{\left\|H\right\|\to 0}{\longrightarrow}0$

So $g$ is differentiable at $X$ and $D_X(H)=-X^{-1}HX^{-1}$

---

$h$ is linear and hence differential, its derivative being itself.

---

Now just use the chain rule and you've got your result.

Answer 2

This is a shorter argument. Less rigorous, but it's easy to follow.

$$ (X + \delta X)(X^{-1} + \delta X^{-1}) = I $$ $$ I + (\delta X) X^{-1} + X (\delta X^{-1}) + \text{(higher order terms)} = I $$ $$ \delta X^{-1} = - X^{-1} (\delta X) X^{-1} + \text{(higher order terms)}.$$