3
$\begingroup$

Dear Matrix Calculus experts, please do enlighten me. How to arrive at the following matrix derivative?

$$\frac{\partial }{\partial X} \log( \mathrm{det}(I + A X B^\top ) ) = A^\top (I + B X^\top A^\top )^{-1} B$$

I am in a confused state with matrix calculus because I am learning and not fully grasping the concept, I must admit. In Wikipedia, it says the following:

It is often easier to work in differential form and then convert back to normal derivatives.

My source of confusion begins from here. So, the differential is

$$\mathrm d \log(\det(X)) = \mbox{tr} \left( X^{-1} \mathrm d X \right)$$

now do we "convert" it to normal derivative such that I get my above answer? :/ I hope I will get this matrix calculus someday. Thank you so much in advance.

$\endgroup$
4
$\begingroup$

Rather than the trace function, the trace/Frobenius product notation $$A:B={\rm tr}(A^TB)$$ lends itself more readily to algebraic manipulations.

Start with the differential result that you quoted, but in terms of the variable $$Y=(I+AXB^T)$$ instead of $X$. Then change the variable from $Y\rightarrow X$ $$\eqalign{ d\log\det Y &= Y^{-T}:dY\cr &= Y^{-T}:A\,dX\,B^T \cr &= A^TY^{-T}B:dX \cr &= A^T(I+BX^TA^T)^{-1}B:dX\cr }$$ The gradient is therefore $$\eqalign{ \frac{\partial\log\det Y}{\partial X} &= A^T(I+BX^TA^T)^{-1}B \cr\cr }$$ Note that the cyclic property of the trace function gives rise to many equivalent ways to arrange the terms in a Frobenius product, e.g. $$\eqalign{ A:BC &= BC:A \cr &= A^T:(BC)^T \cr &= AC^T:B \cr &= B^TA:C \cr &= B^TAC^T:I \cr &= etc \cr }$$

$\endgroup$
  • $\begingroup$ Thanks greg. I really like your approach. must say that. Thank you so much. I have one clarification for now. Is the differential of $\log \det (X) = X^{-T} : dX$ or $\log \det (X) = X^{-1} : dX$? $\endgroup$ – user550103 Jun 16 '18 at 14:35
  • $\begingroup$ I mean the following differential form you have mentioned: $d\log\det Y= Y^{-T}:dY$ is not due to the cyclic property of $\rm tr$ function, is it? $\endgroup$ – user550103 Jun 16 '18 at 15:08
  • $\begingroup$ @user550103 It's the same formula as you quoted, just reformatted as a Frobenius product $${\rm tr}(X^{-1}dX)=X^{-T}:dX$$ $\endgroup$ – greg Jun 16 '18 at 16:55
  • $\begingroup$ Thank you for the clarification. $\endgroup$ – user550103 Jun 17 '18 at 6:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.