2
$\begingroup$

I am having trouble differentiating $\frac {\partial(tr(AX^{-1}))}{\partial X}$. If I understand, how chain rule works in matrix derivation (if it works at all), then I'm getting this result: $$\frac {\partial(tr(AX^{-1}))}{\partial X} = \frac {\partial(tr(AX^{-1}))}{\partial (X^{-1})} \frac {\partial(X^{-1})}{\partial X}$$ which, using matrix cookbook, particularly equations (101) and (59), gives me $$\frac {\partial(tr(AX^{-1}))}{\partial (X^{-1})} \frac {\partial(X^{-1})}{\partial X} = A^TX^{-2}$$

This is the answer I get, but I'm not sure at all that that is how it works.

$\endgroup$
  • $\begingroup$ a particular case is easier : if $ \|Y\| < 1$ (the operator norm $\|Y\| = \max_{\|v\|=1} \|Y v\|$) then the series $\sum_{k=0}^\infty Y^k$ converges and is equal to $(I-Y)^{-1}$. hence $\frac{\partial (tr(A (I-Y)^{-1}))}{\partial Y} = ?$ $\endgroup$ – reuns Apr 17 '16 at 15:53
  • $\begingroup$ @user1952009 I'm sorry, but I don't think I follow, what does this give. Also, I don't have any such guarantees. $\endgroup$ – Alex Apr 17 '16 at 16:02
  • $\begingroup$ since $I$ is constant, $\frac{\partial}{\partial (I-Y)}$ and $-\frac{\partial}{\partial Y}$ is the same. hence if $X = I-Y$ with $\|Y\| < 1$ ... and what I wanted to show you is that even if you are treating matrices, you can write the Taylor expansion $((I-\epsilon Y)M)^{-1} = M^{-1} (I-\epsilon Y)^{-1} = M^{-1} \sum_{k=0}^\infty \epsilon^k Y^k = M^{-1} + \epsilon M^{-1} Y + \mathcal{O}(\epsilon^2)$ $\endgroup$ – reuns Apr 17 '16 at 16:12
2
$\begingroup$

Write the function in terms of the Frobenius (:) Inner Product and take its differential $$\eqalign{ f &= {\rm tr}(AX^{-1}) = A^T: X^{-1} \cr\cr df &= A^T: dX^{-1} \cr &= -A^T: X^{-1}\,dX\,X^{-1} \cr &= -X^{-T}A^TX^{-T}:dX \cr }$$ Since $df=\big(\frac{\partial f}{\partial X}:dX\big),\,$ the gradient must be $$\eqalign{ \frac{\partial f}{\partial X} &= -X^{-T}A^TX^{-T} \cr }$$

$\endgroup$
  • $\begingroup$ Thanks for the help. I'll be honest, I've never encountered Frobenius inner product. $\endgroup$ – Alex Apr 17 '16 at 18:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.