Let A be a constant matrix. Define $$ Y:= \mathrm{tr}(X^\top A) A $$

I want to find \begin{align} \frac{\partial Y}{\partial X} \label{A} \end{align}

I know that $$ \frac{\partial\,\mathrm{tr}(X^\top A)}{\partial X} = A \label{B} $$

Of course, I should be able to extend this to the case I want. But I am a bit confused as to how I should notate the result. What I did to find \eqref{A} was to vectorize both sides and then use the fact in \eqref{B}.

$$ \mathrm{vec(Y)} = \mathrm{tr}(X^\top A)\mathrm{vec}(A) $$

I also knwo that I can write $$ \mathrm{tr}(X^\top A) = \mathrm{vec}(A)^\top \mathrm{vec}(X) $$ So the desired the derivative should be $$ \mathrm{vec}(A) \otimes \mathrm{vec}(A) $$ Is this correct?.

  • $\begingroup$ The problem can be approached in either tensor or vector form $$\eqalign{&Y=(A\star A):X&\implies\,y=aa^Tx\cr &\frac{\partial Y}{\partial X}=(A\star A)&\implies\,\frac{\partial y}{\partial x}=aa^T}$$where $a={\rm vec}(A),\,x={\rm vec}(X),$ and $\star$ is the tensor product $${\mathcal A}=B\star C \implies {\mathcal A}_{ijkl}=B_{ij}C_{kl}$$ $\endgroup$ – greg Apr 29 '18 at 14:09

Your result seems correct. You may verify it by using the entry-wise form.

Let the entry-wise form of the matrices $A$, $X$ and $Y$ be \begin{align} A&=\left(a_{jk}\right)_{jk},\\ X&=\left(x_{jk}\right)_{jk},\\ Y&=\left(y_{jk}\right)_{jk}. \end{align} Then $Y=\text{tr}\left(X^{\top}A\right)A$ reads $$ y_{jk}=\left(\sum_{i,r}x_{ir}a_{ir}\right)a_{jk}. $$

With this notation, \begin{align} \frac{\partial y_{jk}}{\partial x_{pq}}&=\frac{\partial}{\partial x_{pq}}\left[\left(\sum_{i,r}x_{ir}a_{ir}\right)a_{jk}\right]\\ &=\sum_{i,r}\frac{\partial x_{ir}}{\partial x_{pq}}a_{ir}a_{jk}\\ &=\sum_{i,r}\delta_{ip}\delta_{rq}a_{ir}a_{jk}\\ &=a_{pq}a_{jk}. \end{align} Thus if one regards $A$ as a tensor, one may write $$ \frac{\partial Y}{\partial X}=A\otimes A. $$

Alternatively, one may use the differential form $$ {\rm d}y_{jk}=\left(\sum_{p,q}a_{pq}{\rm d}x_{pq}\right)a_{jk}, $$ which, in its matrix form, reads $$ {\rm d}Y=\text{tr}\left(A^{\top}{\rm d}X\right)A. $$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.