Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

How to do the derivative \begin{equation} \frac{ \partial {\mathrm{tr}(XX^TXX^T)}}{\partial X}\quad ? \end{equation}

I have no idea where to start.

share|cite|improve this question
What does $\frac{\partial}{\partial X}$ mean if $X$ is a matrix? – user27126 Dec 14 '12 at 7:35
You could start here – Learner Dec 14 '12 at 7:35
@Learner Thanks but is there any trick I can use if the function is $tr(XX^TXX^T)$? – Rein Dec 14 '12 at 7:39
@Rein, the derivative is really the "linear part" of a function, so if you want to take the derivative at $A$, write $X = A+H$, and capture the linear part involving $H$. – user27126 Dec 14 '12 at 7:41
up vote 0 down vote accepted

Let's $F(X)=tr(XX^TXX^T)$. We have

\begin{align} F(X+H)= & tr\bigg([X+H][X+H]^T[X+H][X+H]^T\bigg) \\ = & tr\bigg(XX^TXX^T \bigg) \\ + & tr\bigg(XX^TXH^T\bigg)+ tr\bigg(XX^THX^T\bigg) \\ + & tr\bigg(XH^TXX^T \bigg)+ tr\bigg(HX^TXX^T \bigg) \\ + & \|H\|\cdot O(H) \end{align} Here $\|H\|=\sqrt{tr(HH^T)}$ is the Frobenius norm and $\displaystyle\lim_{H\to 0}O(H)=0$. Then the total derivative is \begin{align} \mathcal{D}F(X)\cdot H = & tr\bigg(XX^TXH^T\bigg)+ tr\bigg(XX^THX^T\bigg) \\ + & tr\bigg(XH^TXX^T \bigg)+ tr\bigg(HX^TXX^T \bigg). \\ \end{align}

The directional derivative is $$ \frac{\partial}{\partial V}F(X)=\mathcal{D}F(X)\cdot V $$ and the partial derivative is $$ \frac{\partial}{\partial E_{ij}}F(X)=\mathcal{D}F(X)\cdot E_{ij}. $$ Here $E_{ij}=[\delta_{ij}]_{n\times m}$.

share|cite|improve this answer

Write it down in terms of components. You want to know $$\frac{\partial}{\partial X_{ij}} \mathop{\rm tr}(X X^T X X^T) =\frac{\partial}{\partial X_{ij}} ( X_{kl} X_{ml} X_{mn} X_{kn}), $$ where summation over repeated indices is implied. Using the fact that $$\frac{\partial}{\partial X_{ij}} X_{mn} =\delta_{im} \delta_{jn}$$ yields $$\begin{align}\frac{\partial}{\partial X_{ij}} \mathop{\rm tr}(X X^T X X^T) &= \delta_{ik} \delta_{jl} X_{ml} X_{mn} X_{kn} + \delta_{im} \delta_{jl} X_{kl} X_{mn} X_{kn} + \delta_{im} \delta_{jn} X_{kl} X_{ml} X_{kn} + \delta_{ik} \delta_{jn} X_{kl} X_{ml} X_{mn} \\ &= X_{mj} X_{mn} X_{in}+X_{kj} X_{in} X_{kn} + X_{kl} X_{il} X_{kj} +X_{il} X_{ml} X_{mj}\\ &=4(X X^T X)_{ij} . \end{align}$$

Or in short notation $$\frac{\partial}{\partial X} \mathop{\rm tr}(X X^T X X^T) = 4 X X^T X.$$

share|cite|improve this answer
You are assuming that the matrices are square and commute? We can not assume these properties. – MathOverview Dec 14 '12 at 10:03
@Elias: I did not assume any of those. Or where do you spot that I assume something like that? – Fabian Dec 14 '12 at 13:47

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.