How to do $\frac{ \partial { \mathrm{tr}(XX^TXX^T)}}{\partial X}$

How to do the derivative $$\frac{ \partial {\mathrm{tr}(XX^TXX^T)}}{\partial X}\quad ?$$

I have no idea where to start.

-
What does $\frac{\partial}{\partial X}$ mean if $X$ is a matrix? – user27126 Dec 14 '12 at 7:35
You could start here en.wikipedia.org/wiki/Matrix_calculus#Derivatives_with_matrices – Learner Dec 14 '12 at 7:35
@Learner Thanks but is there any trick I can use if the function is $tr(XX^TXX^T)$? – Rein Dec 14 '12 at 7:39
@Rein, the derivative is really the "linear part" of a function, so if you want to take the derivative at $A$, write $X = A+H$, and capture the linear part involving $H$. – user27126 Dec 14 '12 at 7:41

2 Answers

Let's $F(X)=tr(XX^TXX^T)$. We have

\begin{align} F(X+H)= & tr\bigg([X+H][X+H]^T[X+H][X+H]^T\bigg) \\ = & tr\bigg(XX^TXX^T \bigg) \\ + & tr\bigg(XX^TXH^T\bigg)+ tr\bigg(XX^THX^T\bigg) \\ + & tr\bigg(XH^TXX^T \bigg)+ tr\bigg(HX^TXX^T \bigg) \\ + & \|H\|\cdot O(H) \end{align} Here $\|H\|=\sqrt{tr(HH^T)}$ is the Frobenius norm and $\displaystyle\lim_{H\to 0}O(H)=0$. Then the total derivative is \begin{align} \mathcal{D}F(X)\cdot H = & tr\bigg(XX^TXH^T\bigg)+ tr\bigg(XX^THX^T\bigg) \\ + & tr\bigg(XH^TXX^T \bigg)+ tr\bigg(HX^TXX^T \bigg). \\ \end{align}

The directional derivative is $$\frac{\partial}{\partial V}F(X)=\mathcal{D}F(X)\cdot V$$ and the partial derivative is $$\frac{\partial}{\partial E_{ij}}F(X)=\mathcal{D}F(X)\cdot E_{ij}.$$ Here $E_{ij}=[\delta_{ij}]_{n\times m}$.

-

Write it down in terms of components. You want to know $$\frac{\partial}{\partial X_{ij}} \mathop{\rm tr}(X X^T X X^T) =\frac{\partial}{\partial X_{ij}} ( X_{kl} X_{ml} X_{mn} X_{kn}),$$ where summation over repeated indices is implied. Using the fact that $$\frac{\partial}{\partial X_{ij}} X_{mn} =\delta_{im} \delta_{jn}$$ yields \begin{align}\frac{\partial}{\partial X_{ij}} \mathop{\rm tr}(X X^T X X^T) &= \delta_{ik} \delta_{jl} X_{ml} X_{mn} X_{kn} + \delta_{im} \delta_{jl} X_{kl} X_{mn} X_{kn} + \delta_{im} \delta_{jn} X_{kl} X_{ml} X_{kn} + \delta_{ik} \delta_{jn} X_{kl} X_{ml} X_{mn} \\ &= X_{mj} X_{mn} X_{in}+X_{kj} X_{in} X_{kn} + X_{kl} X_{il} X_{kj} +X_{il} X_{ml} X_{mj}\\ &=4(X X^T X)_{ij} . \end{align}

Or in short notation $$\frac{\partial}{\partial X} \mathop{\rm tr}(X X^T X X^T) = 4 X X^T X.$$

-
You are assuming that the matrices are square and commute? We can not assume these properties. – MathOverview Dec 14 '12 at 10:03
@Elias: I did not assume any of those. Or where do you spot that I assume something like that? – Fabian Dec 14 '12 at 13:47