3
$\begingroup$

I would like to compute the derivative of $\mathrm{trace}((S^T S)^{-2})$ w.r.t. $S$.

I know that $\frac{\partial \mathrm{trace}((S^T S)^{-1})}{\partial S} = -2S(S^T S)^{-2}$ but I have a higher order in my expression.

Any help would be really appreciated.

$\endgroup$
0

2 Answers 2

2
$\begingroup$

Let's use definition $$ \begin{align*} &Tr[((S^T+\Delta^T)(S+\Delta))^{-2}] - Tr[(S^TS)^{-2}] \\ &=Tr[(S^TS+S^T\Delta+\Delta^TS+\Delta^T\Delta)^{-2} - (S^TS)^{-2}]\\ &=Tr[((S^TS)^{-1} - (S^TS)^{-1}(S^T\Delta+\Delta^TS)(S^TS)^{-1} + \mathcal{O}\Big(||\Delta||^2\Big))^2- (S^TS)^{-2}]\\ &=Tr[(S^TS)^{-2}-(S^TS)^{-2}(S^T\Delta+\Delta^TS)(S^TS)^{-1}-(S^TS)^{-1}(S^T\Delta+\Delta^TS)(S^TS)^{-2} - (S^TS)^{-2} + \mathcal{O}\Big(||\Delta||^2\Big)]\\ &=-4Tr[(S^TS)^{-3}S^T\Delta] + \mathcal{O}\Big(||\Delta||^2\Big)\\ &=\langle-4S(S^TS)^{-3},\Delta\rangle + \mathcal{O}\Big(||\Delta||^2\Big) \end{align*} $$

$\endgroup$
7
  • $\begingroup$ Is that the solution for $\frac{\partial \mathrm{trace}((S^T S)^{-2})}{\partial S}$. your solution looks like mine which is for $\frac{\partial \mathrm{trace}((S^T S)^{-1})}{\partial S}$ and not for my question. $\endgroup$
    – user2987
    Feb 26, 2016 at 21:56
  • $\begingroup$ No. It is for $\frac{\partial \mathrm{trace}((S^T S)^{-1})}{\partial S}$ $\endgroup$
    – echzhen
    Feb 26, 2016 at 21:57
  • $\begingroup$ Ah, my bad! I will try to do it for -2 $\endgroup$
    – echzhen
    Feb 26, 2016 at 21:58
  • $\begingroup$ thanks a lot. I would really appreciate it. $\endgroup$
    – user2987
    Feb 26, 2016 at 22:00
  • 2
    $\begingroup$ @user2987 see update, seems that it's nothing but chain rule. And in general you will have $\partial_{S}Tr[(S^TS)^{-n}] = -2nS(S^TS)^{-n-1}$ for $n>0$ $\endgroup$
    – echzhen
    Feb 26, 2016 at 22:07
2
$\begingroup$

Define a new symmetric matrix variable $$\eqalign{ X &= (S^TS)^{-1} \cr dX &= -X\,\,d\big(S^T\!S\big)\,\,X \cr &= -X\,\,(dS^TS+S^TdS)\,\,X \cr &= -2\,X\,\,{\rm sym}(S^TdS)\,\,X \cr }$$ Write the function using the Frobenius Inner Product and this new variable. Then finding the differential and gradient is pretty easy. $$\eqalign{ f &= X:X \cr\cr df &= 2\,X:dX \cr &= -4\,X:X\,{\rm sym}(S^T\,dS)\,X \cr &= -4\,X^3:{\rm sym}(S^T\,dS) \cr &= -4\,{\rm sym}(X^3):S^T\,dS \cr &= -4\,X^3:S^T\,dS \cr &= -4\,SX^3:dS \cr\cr \frac{\partial f}{\partial S} &= -4\,SX^3 \cr &= -4\,S(S^TS)^{-3} \cr }$$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.