Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

The problem is $\frac {\partial \mathrm{tr}(Q^TQAQ^TQA)}{\partial q_i}$, where $Q=[q_1,...,q_N]$, $q_i$ is $N$ dimensional vector and $Q$ is $N\times N$ matrix.

I have think of using chain rule, but I am confusing on using chain rule on matrix calculus.

For example if we let $X=Q^TQA$, the problem becomes $\frac {\partial \mathrm{tr}(X^2)}{\partial q_i}$, if I use the chain rule in scalar differentiation it will becomes $\frac {\partial \mathrm{tr}(X^2)}{\partial X} $ $\frac {\partial X}{\partial q_i}$ and it seems to be invalid.

share|improve this question
    
Why you think that the chain rule is invalid? –  Fabian Dec 28 '12 at 7:24
    
@Fabian Because I don't know how to handle $\frac {\partial X} {\partial q_i}$ –  Rein Dec 28 '12 at 7:27
    
Are the $q_i$ the rows or the columns of $Q$? –  Fabian Dec 28 '12 at 7:29
    
@Fabian They are columns of $Q$ –  Rein Dec 28 '12 at 7:36
add comment

2 Answers

up vote 1 down vote accepted

I do not get the argument why the chain rule is invalid. You can in fact proceed using the chain rule. For such questions is it usually easiest to write everything out in components. So first, we need $$\partial_{X_{mn}} \mathop{\rm tr}X^2 =\partial_{X_{mn}} X_{ij}X_{ji} = \delta_{mi} \delta_{nj} X_{ji}+ \delta_{mj} \delta_{ni} X_{ij} = 2 X_{nm}; $$ here, we have assumed that all indices which appear twice are summed over and used the fact that $\partial_{X_{mn}} X_{ij} = \delta_{mi} \delta_{nj}$.

Next, we need $$\begin{align}\partial_{(q_i)_j} X_{mn} &=\partial_{(q_i)_j} Q_{km} Q_{kl} A_{ln} =\partial_{(q_i)_j} (q_m)_k (q_l)_k A_{ln} = \delta_{im} \delta_{jk} (q_l)_k A_{ln} + \delta_{il} \delta_{jk} (q_m)_k A_{ln}\\ &=(q_l)_j A_{ln} +(q_m)_j A_{in}\\ &= Q_{jl} A_{ln} + Q_{jm} A_{in} \end{align}$$ because $Q_{mn} = (q_n)_m$.

In conclusion, we have $$\begin{align}\partial_{(q_i)_j} \mathop{\rm tr}(Q^TQAQ^TQA) &= \partial_{X_{mn}} \mathop{\rm tr}X^2 \partial_{(q_i)_j} X_{mn} = 2 X_{mn} [ Q_{jl} A_{ln} + Q_{jm} A_{in}]\\ &= 2 Q_{km} Q_{kl} A_{ln}^2 Q_{jl} + 2Q_{jm} Q_{km} Q_{kl} A_{ln} A_{in}\\ &= 2(Q Q^T Q B \mathop{\rm tr} (A A^T) + Q Q^T Q A A^T)_{ji} \end{align}$$ with $(B)_{ij}= 1$ the constant unit matrix.

share|improve this answer
    
You inexplicably dropped $\delta_{im}$ in your expansion of $\partial_{(q_i)_j} X_{mn}$. That result should read $(\delta_{im} Q_{jl} A_{ln} + Q_{jm} A_{in})$. Which changes the conclusion to $2(A^T Q^T Q A^T + A Q^T Q A) Q^T$. –  lynne Feb 18 at 4:03
add comment

Let's a function $F:M_{N\times N}(\mathbb{R})\to \mathbb{R}$ definid by $F(Q)=\mathrm{tr}\Big(Q^TQAQ^TQA\Big)$ then $$ \frac{\partial}{\partial q_i}\mathrm{tr}\Big(Q^TQAQ^TQA\Big)=\mathcal{D} F(Q)\cdot[0\ldots q_i\ldots 0]. $$ Here $$ [0\ldots q_i\ldots 0] = \begin{pmatrix} 0&\dots &q_{1i}&\dots & 0 \\ \vdots & \cdots & \vdots & \cdots & \vdots \\ 0&\dots &q_{ii}&\dots &0 \\ \vdots & \cdots & \vdots & \cdots & \vdots \\ 0&\dots &q_{Ni}&\dots &0 \\ \end{pmatrix} \mbox{ and } q_i= \begin{pmatrix} q_{1i} \\ \vdots \\ q_{ii} \\ \vdots \\ q_{Ni} \end{pmatrix} $$ and $\mathcal{D}F(Q_0): M_{N\times N}(\mathbb{R})\to \mathbb{R}$ is the total derivative of $F$ at $Q_0$, i.e. $$ F(Q_0+V)=F(Q_0)+\mathcal{D}F(Q_0)\cdot V+ \|V\|\cdot\rho(V),\quad \lim_{V\to 0}\frac{\rho(V)}{\|V\|}=0\quad \mbox{ and }\|V\|=\sqrt{tr(V^TV)} $$ Note that \begin{align} F(Q+V) = & tr\Big([Q+V][Q+V]^TA[Q+V]^T[Q+V]A\Big) \\ = & tr(QQ^TAQ^TQA)+ \\ & \\ & +tr(VQ^TAQ^TQA)+tr(QV^TAQ^TQA)+ \\ & +tr(QQ^TAV^TQA)+tr(QQ^TAQ^TVA)+ \\ & \\ & +tr(VV^TAQ^TQA)+ \\ & +tr(VQ^TAV^TQA)+tr(VQ^TAQ^TVA)+ \\ & +tr(QV^TAV^TQA)+tr(QV^TAQ^TVA)+ \\ & +tr(QQ^TAV^TVA)+ \\ & \\ & +tr(QV^TAV^TVA)+tr(VQ^TAV^TVA)+ \\ & +tr(VV^TAQ^TVA)+tr(VV^TAV^TQA)+ \\ & \\ & +tr(VV^TAV^TVA) \end{align} implies \begin{align} \mathcal{D}F(Q)\cdot V= & tr(VQ^TAQ^TQA)+tr(QV^TAQ^TQA) \\ & +tr(QQ^TAV^TQA)+tr(QQ^TAQ^TVA). \end{align} Play the matrix $V$ by matrix$[0,\ldots,q_i,\ldots,0]$ and calculate the answer. Good Look.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.