How to differentiate inverse of a matrix inside frobenius norm? Basically, I have an equation in the form
$$
f = \left \| A^{-1} \right \|^2
$$
I need to differentiate the above equation wrt A.
The matrix A is just a 2X2 matrix so I tried to solve it by brute force. It worked but the solution is very long. Not to mention, it is very time-consuming. I was wondering if there is a general method to solve this problem that will work for any order of matrix A.
 A: Wikipedia tells us: $$\|M\|=\sqrt{\mathrm{tr}(A^*A)}$$  Thus \begin{align*}
\|A+tX\|^2&=\mathrm{tr}(A^*A+tX^*A+tA^*X+t^2X^*X) \\
&=\mathrm{tr}(A^*A)+2t\,\mathrm{tr}(X^*A)+t^2\mathrm{tr}(X^*X)
\end{align*} by linearity, conjugation invariance, and cyclic invariance.  Looking for the linear term, $$D_A{(\|A\|^2)}(X)=2\mathrm{tr}(X^*A)$$  Likewise, $$D_A(A^{-1})(X)=-A^{-1}XA^{-1}$$  Thus by the chain rule, $$D_A{(\|A^{-1}\|^2)}(X)=2\mathrm{tr}((-A^{-1}XA^{-1})^*A^{-1})$$
A: $
\def\A{A^{-1}}
\def\At{A^{-T}}
\def\p{\partial}
\def\L{\left}\def\R{\right}\def\LR#1{\L(#1\R)}
\def\trace#1{\operatorname{Tr}\LR{#1}}
\def\qiq{\quad\implies\quad}
\def\grad#1#2{\frac{\p #1}{\p #2}}
\def\c#1{\color{red}{#1}}
\def\CLR#1{\c{\LR{#1}}}
$The double-dot product provides a concise notation for the trace and the Frobenius norm
$$\eqalign{
A:B &= \sum_{i=1}^m\sum_{j=1}^n A_{ij}B_{ij} \;=\; \trace{A^TB} \\
A:A &= \|A\|^2_F \\
}$$
The properties of the underlying trace function allow the terms in such a product to be rearranged in many different but equivalent ways, e.g.
$$\eqalign{
A:B &= B:A \\
A:B &= A^T:B^T \\
C:\LR{AB} &= \LR{CB^T}:A \\&= \LR{A^TC}:B \\
}$$
Write your function using this notation, then calculate its differential and gradient.
$$\eqalign{
f
 &= \A:\A \\
df
 &= {d\A}:\A \;+\; \A:{d\A} \\
 &= 2\A:\c{d\A} \\
 &= 2\A:\CLR{-\A\,dA\,\A} \\
 &= -2\LR{\At\A\At}:dA \\
 &= -2\LR{A^TAA^T}^{-1}:dA \\
\grad{f}{A}
 &= -2\LR{A^TAA^T}^{-1} \\
}$$
