Finding the second derivative of $f:G \rightarrow L(\mathbb{R}^n,\mathbb{R}^n), f(A) = A^{-1}$ Let $G = \{T \in L(\mathbb{R^n}, \mathbb{R^n}) ; \exists T^{-1} \}$ and $f:G \rightarrow L(\mathbb{R}^n,\mathbb{R}^n), f(A) = A^{-1}$ Find an expression for $f''(A)(H,K)$
Am I supposed to write $f(A+H) = f(A) + T(H) + r(H)$ and manipulate it till I find $T$ linear and $\lim_{h \rightarrow 0} \frac{r(H)}{|H|}$? Because I cant separate $(A+H)^{-1}$...
Thanks
 A: First derivative:
$$f(X+H)=(X+H)^{-1}=(X(I+X^{-1}H))^{-1}=(I+X^{-1}H)^{-1}X^{-1}=(I-(-X^{-1}H))^{-1}X^{-1}=\big(\sum_{n=0}^{\infty}(-X^{-1}H)^n\big)X^{-1}=X^{-1}- X^{-1}HX^{-1}+o(H).$$
Therefore, $f'_X(\cdot)=-X^{-1}(\cdot)X^{-1}$.
Second derivative:
$$f'_{X+H}=-(X+H)^{-1}(\cdot) (X+H)^{-1}=(-X^{-1}+X^{-1}HX^{-1}+o(H))(\cdot)(X^{-1}-X^{-1}HX^{-1}+o(H))$$
$$=-X^{-1}(\cdot)X^{-1}+X^{-1}HX^{-1}(\cdot)X^{-1}+X^{-1}(\cdot)X^{-1}HX^{-1}+o(H).$$
Therefore,
$$f''_{X}(H)=X^{-1}HX^{-1}(\cdot)X^{-1}+X^{-1}(\cdot)X^{-1}HX^{-1}.$$
Identifying $f''$ with the bilinear application, we have
$$f''(A)(H,K)=A^{-1}HA^{-1}KA^{-1}+A^{-1}KA^{-1}HA^{-1}.$$
A: I might not be parsing the question correctly, but here's an alternative approach to calculating partial derivatives of $A^{-1}$ as a function of the coordinates of $A$ (which can themselves be further dependent on other parameters if you'd like).
Let $A \in GL_n(\mathbb{R})$, the group of invertible $n \times n$ matrices with real coefficients and suppose $A^{-1}$. Suppose $A$ is a function of the variables $x = (x_1,\ldots,x_m)$. (For example, you can take $m = n^2$ and $x$ to be a list of the components of $A$.) Fix a multi-index $\alpha = (\alpha_1,\ldots,\alpha_m)$. The key idea here is that if $A$ is $\alpha$-times differentiable with respect to $x$, then so is $A^{-1}$ (since its components are rational functions in the coordinates of $A$ that are defined at the values of the components of $A$), and so we can differentiate the equation $AA^{-1} = 1_n$ using the product rule to obtain
$$
    \frac{\partial A}{\partial x^\alpha}A^{-1} + A\frac{\partial A^{-1}}{\partial x^\alpha} = 0.
$$
(If you haven't derived the product rule for matrices before, it can be done component-wise.) Rearranging the last equation yields
$$
    A\frac{\partial A^{-1}}{\partial x^\alpha} = -\frac{\partial A}{\partial x^\alpha}A^{-1},
$$
and (hence)
$$
    \frac{\partial A^{-1}}{\partial x^\alpha} = -A^{-1}\frac{\partial A}{\partial x^\alpha}A^{-1}.
$$
A: You can find a formula for $(A+H)^{-1}$ by observing that $(A+H)A^{-1}\sum_{k=0}^{\infty }(-1)^{k}\left ( HA^{-1} \right )^{k}=I$, which will be valid for $\left \| H \right \|$ small enough.
