Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

How to do the derivative \begin{equation} \frac{ \partial {\mathrm{tr}(XX^TXX^T)}}{\partial X}\quad ? \end{equation}

I have no idea where to start.

share|improve this question
    
What does $\frac{\partial}{\partial X}$ mean if $X$ is a matrix? –  user27126 Dec 14 '12 at 7:35
1  
You could start here en.wikipedia.org/wiki/Matrix_calculus#Derivatives_with_matrices –  Learner Dec 14 '12 at 7:35
    
@Learner Thanks but is there any trick I can use if the function is $tr(XX^TXX^T)$? –  Rein Dec 14 '12 at 7:39
    
@Rein, the derivative is really the "linear part" of a function, so if you want to take the derivative at $A$, write $X = A+H$, and capture the linear part involving $H$. –  user27126 Dec 14 '12 at 7:41

2 Answers 2

up vote 0 down vote accepted

Let's $F(X)=tr(XX^TXX^T)$. We have

\begin{align} F(X+H)= & tr\bigg([X+H][X+H]^T[X+H][X+H]^T\bigg) \\ = & tr\bigg(XX^TXX^T \bigg) \\ + & tr\bigg(XX^TXH^T\bigg)+ tr\bigg(XX^THX^T\bigg) \\ + & tr\bigg(XH^TXX^T \bigg)+ tr\bigg(HX^TXX^T \bigg) \\ + & \|H\|\cdot O(H) \end{align} Here $\|H\|=\sqrt{tr(HH^T)}$ is the Frobenius norm and $\displaystyle\lim_{H\to 0}O(H)=0$. Then the total derivative is \begin{align} \mathcal{D}F(X)\cdot H = & tr\bigg(XX^TXH^T\bigg)+ tr\bigg(XX^THX^T\bigg) \\ + & tr\bigg(XH^TXX^T \bigg)+ tr\bigg(HX^TXX^T \bigg). \\ \end{align}

The directional derivative is $$ \frac{\partial}{\partial V}F(X)=\mathcal{D}F(X)\cdot V $$ and the partial derivative is $$ \frac{\partial}{\partial E_{ij}}F(X)=\mathcal{D}F(X)\cdot E_{ij}. $$ Here $E_{ij}=[\delta_{ij}]_{n\times m}$.

share|improve this answer

Write it down in terms of components. You want to know $$\frac{\partial}{\partial X_{ij}} \mathop{\rm tr}(X X^T X X^T) =\frac{\partial}{\partial X_{ij}} ( X_{kl} X_{ml} X_{mn} X_{kn}), $$ where summation over repeated indices is implied. Using the fact that $$\frac{\partial}{\partial X_{ij}} X_{mn} =\delta_{im} \delta_{jn}$$ yields $$\begin{align}\frac{\partial}{\partial X_{ij}} \mathop{\rm tr}(X X^T X X^T) &= \delta_{ik} \delta_{jl} X_{ml} X_{mn} X_{kn} + \delta_{im} \delta_{jl} X_{kl} X_{mn} X_{kn} + \delta_{im} \delta_{jn} X_{kl} X_{ml} X_{kn} + \delta_{ik} \delta_{jn} X_{kl} X_{ml} X_{mn} \\ &= X_{mj} X_{mn} X_{in}+X_{kj} X_{in} X_{kn} + X_{kl} X_{il} X_{kj} +X_{il} X_{ml} X_{mj}\\ &=4(X X^T X)_{ij} . \end{align}$$

Or in short notation $$\frac{\partial}{\partial X} \mathop{\rm tr}(X X^T X X^T) = 4 X X^T X.$$

share|improve this answer
    
You are assuming that the matrices are square and commute? We can not assume these properties. –  Elias Dec 14 '12 at 10:03
    
@Elias: I did not assume any of those. Or where do you spot that I assume something like that? –  Fabian Dec 14 '12 at 13:47

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.