Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It only takes a minute to sign up.
Sign up to join this community
I would like to compute the derivative of $\mathrm{trace}((S^T S)^{-2})$ w.r.t. $S$.
I know that $\frac{\partial \mathrm{trace}((S^T S)^{-1})}{\partial S} = -2S(S^T S)^{-2}$ but I have a higher order in my expression.
Any help would be really appreciated.
Let's use definition $$ \begin{align*} &Tr[((S^T+\Delta^T)(S+\Delta))^{-2}] - Tr[(S^TS)^{-2}] \\ &=Tr[(S^TS+S^T\Delta+\Delta^TS+\Delta^T\Delta)^{-2} - (S^TS)^{-2}]\\ &=Tr[((S^TS)^{-1} - (S^TS)^{-1}(S^T\Delta+\Delta^TS)(S^TS)^{-1} + \mathcal{O}\Big(||\Delta||^2\Big))^2- (S^TS)^{-2}]\\ &=Tr[(S^TS)^{-2}-(S^TS)^{-2}(S^T\Delta+\Delta^TS)(S^TS)^{-1}-(S^TS)^{-1}(S^T\Delta+\Delta^TS)(S^TS)^{-2} - (S^TS)^{-2} + \mathcal{O}\Big(||\Delta||^2\Big)]\\ &=-4Tr[(S^TS)^{-3}S^T\Delta] + \mathcal{O}\Big(||\Delta||^2\Big)\\ &=\langle-4S(S^TS)^{-3},\Delta\rangle + \mathcal{O}\Big(||\Delta||^2\Big) \end{align*} $$
Define a new symmetric matrix variable $$\eqalign{ X &= (S^TS)^{-1} \cr dX &= -X\,\,d\big(S^T\!S\big)\,\,X \cr &= -X\,\,(dS^TS+S^TdS)\,\,X \cr &= -2\,X\,\,{\rm sym}(S^TdS)\,\,X \cr }$$ Write the function using the Frobenius Inner Product and this new variable. Then finding the differential and gradient is pretty easy. $$\eqalign{ f &= X:X \cr\cr df &= 2\,X:dX \cr &= -4\,X:X\,{\rm sym}(S^T\,dS)\,X \cr &= -4\,X^3:{\rm sym}(S^T\,dS) \cr &= -4\,{\rm sym}(X^3):S^T\,dS \cr &= -4\,X^3:S^T\,dS \cr &= -4\,SX^3:dS \cr\cr \frac{\partial f}{\partial S} &= -4\,SX^3 \cr &= -4\,S(S^TS)^{-3} \cr }$$
