$\frac{\partial X^{-1}_{mn}}{\partial X_{ij}}$? What is the derivative of the $(m,n)^{th}$ element of the inverse, $X^{-1}$, with respect to the $(i,j)^{th}$ element of $X$?
i.e. what is;
$$\frac{\partial X^{-1}_{mn}}{\partial X_{ij}}$$
Is it true that;
$$\frac{\partial A^{-1}_{mn}}{\partial A_{ij}}=-\sum_k \sum_l A^{-1}_{mk} \frac{\partial A_{kl}}{\partial A_{ij}} A^{-1}_{ln}=-A^{-1}_{mi}A^{-1}_{jn}$$
 A: $
\def\p{\partial}\def\X{X^{-1}}
\def\L{\left}\def\R{\right}\def\LR#1{\L(#1\R)}
\def\grad#1#2{\frac{\p #1}{\p #2}}
$Let $e_k$ denote a standard basis vector, whose components are all zero except for the $k^{th}$ element which is equal to ${\tt1}.\,$ Such vectors can be used to write the component-wise self-gradient of a matrix
$$\eqalign{
\grad{X}{X_{ij}} &= e_i\,e_j^T \\
}$$
The component-wise gradient of the inverse can be calculated as
$$\eqalign{
I &= X\X \\
\grad{I}{X_{ij}}
 &= \LR{\grad{X}{X_{ij}}}\X + X\LR{\grad{\X}{X_{ij}}} \\
X\LR{\grad{\X}{X_{ij}}} &= -\LR{\grad{X}{X_{ij}}}\X  \\
\grad{\X}{X_{ij}} &= -\X\LR{e_i\,e_j^T}\X \\
}$$
To extract the $(m,n)$ component multiply by more basis vectors
$$\eqalign{
\grad{\X_{mn}}{X_{ij}}
 &= e_m^T\LR{\grad{\X}{X_{ij}}}e_n \\
 &= -e_m^T\X e_i\;e_j^T\X e_n \\
 &= -\X_{mi}\;\X_{jn} \\
}$$
which confirms the formula.
A: Another related approach is as follows. Consider the Frobenius scalar product
$\phi=\mathbf{e}_m : \mathbf{X}^{-1} \mathbf{e}_n$
The differential writes
\begin{eqnarray}
d\phi
&=& -\mathbf{e}_m: \mathbf{X}^{-1} (d\mathbf{X}) \mathbf{X}^{-1} \mathbf{e}_n \\
&=& - \mathbf{X}^{-T} \mathbf{e}_m \mathbf{e}_n^T \mathbf{X}^{-T} : d\mathbf{X}  \\
\end{eqnarray}
The LHS term in the scalar product is the (whole) matrix gradient.
If you are looking for the $(i,j)$-th component, the requested quantity is
$- (\mathbf{e}_i^T \mathbf{X}^{-T} \mathbf{e}_m) (\mathbf{e}_n^T \mathbf{X}^{-T} \mathbf{e}_j)$
which is exactly what you obtained.
