What is the derivative of $\mathrm{trace}((S^T S)^{-2})$ w.r.t. $S$ I would like to compute the derivative of $\mathrm{trace}((S^T S)^{-2})$ w.r.t. $S$.
I know that $\frac{\partial \mathrm{trace}((S^T S)^{-1})}{\partial S} = -2S(S^T S)^{-2}$ but I have a higher order in my expression.
Any help would be really appreciated.
 A: Let's use definition
$$
\begin{align*}
&Tr[((S^T+\Delta^T)(S+\Delta))^{-2}] - Tr[(S^TS)^{-2}] \\
&=Tr[(S^TS+S^T\Delta+\Delta^TS+\Delta^T\Delta)^{-2} - (S^TS)^{-2}]\\
&=Tr[((S^TS)^{-1} - (S^TS)^{-1}(S^T\Delta+\Delta^TS)(S^TS)^{-1} + \mathcal{O}\Big(||\Delta||^2\Big))^2- (S^TS)^{-2}]\\
&=Tr[(S^TS)^{-2}-(S^TS)^{-2}(S^T\Delta+\Delta^TS)(S^TS)^{-1}-(S^TS)^{-1}(S^T\Delta+\Delta^TS)(S^TS)^{-2} - (S^TS)^{-2} + \mathcal{O}\Big(||\Delta||^2\Big)]\\
&=-4Tr[(S^TS)^{-3}S^T\Delta] + \mathcal{O}\Big(||\Delta||^2\Big)\\
&=\langle-4S(S^TS)^{-3},\Delta\rangle + \mathcal{O}\Big(||\Delta||^2\Big)
\end{align*}
$$
A: Define a new symmetric matrix variable
$$\eqalign{
 X &= (S^TS)^{-1} \cr
dX &= -X\,\,d\big(S^T\!S\big)\,\,X \cr
   &= -X\,\,(dS^TS+S^TdS)\,\,X \cr
   &= -2\,X\,\,{\rm sym}(S^TdS)\,\,X \cr
}$$
Write the function using the Frobenius Inner Product and this new variable. Then finding the differential and gradient is pretty easy.
$$\eqalign{
 f &= X:X \cr\cr
df &= 2\,X:dX \cr
   &= -4\,X:X\,{\rm sym}(S^T\,dS)\,X \cr
   &= -4\,X^3:{\rm sym}(S^T\,dS) \cr
   &= -4\,{\rm sym}(X^3):S^T\,dS \cr
   &= -4\,X^3:S^T\,dS \cr
   &= -4\,SX^3:dS \cr\cr
\frac{\partial f}{\partial S} &= -4\,SX^3 \cr &= -4\,S(S^TS)^{-3} \cr
}$$
