I am stuck with this (probably simple) derivatives:

$$ \frac{\partial}{\partial X}Tr((A\odot(B^{T}XB))C)\;\;and \;\;\frac{\partial}{\partial X}Tr((A\odot(B^{T}XX^{T}B))C) $$

where $A,B,C$ are constant matrices and $\odot$ is the Hadamard product.

Some help please?



The trace/Frobenius product is defined as $$A:B = {\rm Tr}(AB^T)$$ The cyclic property of the trace allows terms in such a product to be rearranged, e.g. $$A:BC \;=\; AC^T:B \;=\; B^TA:C$$ The Frobenius and Hadamard products commute with themselves and each other. $$\eqalign{ A:B &= B:A \\ A\odot B &= B\odot A \\ C:A\odot B &= C\odot A:B \\ }$$ Consider the following of function of the variable $Y$. $$\eqalign{ \phi &= {\rm Tr}\big((A\odot(B^TYB))C\big) \\ &= (A\odot(B^TYB)):C^T \\ &= (A\odot C^T):B^TYB \\ &= B(A\odot C^T)B^T:Y \\ &= M:Y \\ }$$ where $M,\,$ being a combination of constant matrices, is itself a constant matrix.

In the first case, set $Y=X\,$ then calculate the differential and the gradient. $$\eqalign{ \phi_1 &= M:X \\ d\phi_1 &= M:dX \\ \frac{\partial\phi_1}{\partial X} &= M \\ }$$ In the second case, set $Y=XX^T\,$ then $$\eqalign{ \phi_2 &= M:XX^T \\ d\phi_2 &= M:(dX\,X^T+X\,dX^T) \\ &= (M+M^T):dX\,X^T \\ &= (M+M^T)X:dX \\ \frac{\partial\phi_2}{\partial X} &= (M+M^T)X \\ }$$

| cite | improve this answer | |
  • $\begingroup$ Thank you Greg! Where can I find a nice book that explains in simple way how to use the formalism and rules you applied? $\endgroup$ – Fabio Jun 3 at 15:55
  • 2
    $\begingroup$ @Fabio I think greg would at least recommend "Matrix Differential Calculus with Applications in Statistics and Econometrics" by Magnus and Neudecker. [I must admit that I have to read this book myself though] $\endgroup$ – user550103 Jun 3 at 19:30
  • 1
    $\begingroup$ Yes, Magnus and Neudecker is good; I also like Hjorungnes's "Complex-Valued Matrix Derivatives"; there's also an online PDF file titled "The Matrix Cookbook" compiled by Petersen and Pedersen. $\endgroup$ – greg Jun 3 at 22:08
  • $\begingroup$ Great books thank you guys! $\endgroup$ – Fabio Jun 3 at 22:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.