# Second order derivative of the inverse matrix operator

Here is my problem:

Let $f:Gl_{n}(\mathbb{R})\rightarrow Gl_{n}(\mathbb{R})$ defined by $f(X)=X^{-1}$. Compute $f''(X)(H,K)$.

Can anyone help me in this matter?

I calculated $f'(X).H=-X^{-1}HX^{-1}$... so I tried to use some composition of linear functions but did not find the appropriate functions.

• it is useful to know that $(I-M)^{-1} = \sum_{k=0}^\infty M^k$ whenever $\|M\| < 1$. then write $(X+\epsilon M)^{-1} = X^{-1}(I+\epsilon X M)^{-1} = X^{-1} \sum_{k=0}^\infty (-XM)^k \epsilon^k$ for $\epsilon$ small enough, and you can compute $\frac{\partial^k}{\partial \epsilon^k} f(X+\epsilon M)$ easily – reuns Aug 25 '16 at 21:51

Let $f (\mathrm X) := \mathrm X^{-1}$. Hence,

$$\begin{array}{rl} f (\mathrm X + h \mathrm V) &= (\mathrm X + h \mathrm V)^{-1}\\ &= (\mathrm X (\mathrm I + h \mathrm X^{-1} \mathrm V))^{-1}\\ &= (\mathrm I + h \mathrm X^{-1} \mathrm V)^{-1} \mathrm X^{-1}\\ &\approx (\mathrm I - h \mathrm X^{-1} \mathrm V + h^2 \mathrm X^{-1} \mathrm V \mathrm X^{-1} \mathrm V) \mathrm X^{-1}\\ &= f (\mathrm X) - h \mathrm X^{-1} \mathrm V \mathrm X^{-1} + h^2 \mathrm X^{-1} \mathrm V \mathrm X^{-1} \mathrm V \mathrm X^{-1}\end{array}$$

Thus, the 2nd directional derivative of $f$ in the direction of $\mathrm V$ at $\mathrm X$ is $\color{blue}{2 \mathrm X^{-1} \mathrm V \mathrm X^{-1} \mathrm V \mathrm X^{-1}}$.

Using the Taylor formula, (As Rodrigo de Azevedo and user1952009 did) you can calculate $f''(X)(H,H)$. If you want the gneral formula $f''(X)(H,K)$, then you can calculate the derivative (with respect to $X$, considering $H$ as a fixed vector) of $f'(X)(H)=-X^{-1}HX^{-1}$.

We obtain $f''(X)(H,K)=X^{-1}KX^{-1}HX^{-1}+X^{-1}HX^{-1}KX^{-1}$ (that is the "bilinearization" of the quadratic form $f''(X)(H,H)$).

Let $f (\mathrm X) := \mathrm X^{-1}$. The 1st directional derivative of $f$ in the direction of $\mathrm V_1$ at $\mathrm X$ is $\color{blue}{\mathrm X^{-1} \mathrm V_1 \mathrm X^{-1}}$.

Let $\mathrm G (\mathrm X) := \mathrm X^{-1} \mathrm V_1 \mathrm X^{-1}$. Hence,

$$\mathrm G (\mathrm X + h \mathrm V_2) \approx \mathrm G (\mathrm X) + h (\mathrm X^{-1} \mathrm V_1 \mathrm X^{-1} \mathrm V_2 \mathrm X^{-1} + \mathrm X^{-1} \mathrm V_2 \mathrm X^{-1} \mathrm V_1 \mathrm X^{-1})$$

Thus, the 1st directional derivative of $\mathrm G$ in the direction of $\mathrm V_2$ at $\mathrm X$ is

$$\color{blue}{\mathrm X^{-1} \mathrm V_1 \mathrm X^{-1} \mathrm V_2 \mathrm X^{-1} + \mathrm X^{-1} \mathrm V_2 \mathrm X^{-1} \mathrm V_1 \mathrm X^{-1}}$$

which is the 2nd directional derivative of $f$ in the directions of $\mathrm V_1$ and $\mathrm V_2$ at $\mathrm X$. If $\mathrm V_1 = \mathrm V_2 = \mathrm V$, then the 2nd directional derivative of $f$ in the direction of $\mathrm V$ at $\mathrm X$ boils down to $\color{blue}{2 \mathrm X^{-1} \mathrm V \mathrm X^{-1} \mathrm V \mathrm X^{-1}}$.