Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

The problem is $\frac {\partial \mathrm{tr}(Q^TQAQ^TQA)}{\partial q_i}$, where $Q=[q_1,...,q_N]$, $q_i$ is $N$ dimensional vector and $Q$ is $N\times N$ matrix.

I have think of using chain rule, but I am confusing on using chain rule on matrix calculus.

For example if we let $X=Q^TQA$, the problem becomes $\frac {\partial \mathrm{tr}(X^2)}{\partial q_i}$, if I use the chain rule in scalar differentiation it will becomes $\frac {\partial \mathrm{tr}(X^2)}{\partial X} $ $\frac {\partial X}{\partial q_i}$ and it seems to be invalid.

share|cite|improve this question
Why you think that the chain rule is invalid? – Fabian Dec 28 '12 at 7:24
@Fabian Because I don't know how to handle $\frac {\partial X} {\partial q_i}$ – Rein Dec 28 '12 at 7:27
Are the $q_i$ the rows or the columns of $Q$? – Fabian Dec 28 '12 at 7:29
@Fabian They are columns of $Q$ – Rein Dec 28 '12 at 7:36
up vote 1 down vote accepted

I do not get the argument why the chain rule is invalid. You can in fact proceed using the chain rule. For such questions is it usually easiest to write everything out in components. So first, we need $$\partial_{X_{mn}} \mathop{\rm tr}X^2 =\partial_{X_{mn}} X_{ij}X_{ji} = \delta_{mi} \delta_{nj} X_{ji}+ \delta_{mj} \delta_{ni} X_{ij} = 2 X_{nm}; $$ here, we have assumed that all indices which appear twice are summed over and used the fact that $\partial_{X_{mn}} X_{ij} = \delta_{mi} \delta_{nj}$.

Next, we need $$\begin{align}\partial_{(q_i)_j} X_{mn} &=\partial_{(q_i)_j} Q_{km} Q_{kl} A_{ln} =\partial_{(q_i)_j} (q_m)_k (q_l)_k A_{ln} = \delta_{im} \delta_{jk} (q_l)_k A_{ln} + \delta_{il} \delta_{jk} (q_m)_k A_{ln}\\ &=(q_l)_j A_{ln} +(q_m)_j A_{in}\\ &= Q_{jl} A_{ln} + Q_{jm} A_{in} \end{align}$$ because $Q_{mn} = (q_n)_m$.

In conclusion, we have $$\begin{align}\partial_{(q_i)_j} \mathop{\rm tr}(Q^TQAQ^TQA) &= \partial_{X_{mn}} \mathop{\rm tr}X^2 \partial_{(q_i)_j} X_{mn} = 2 X_{mn} [ Q_{jl} A_{ln} + Q_{jm} A_{in}]\\ &= 2 Q_{km} Q_{kl} A_{ln}^2 Q_{jl} + 2Q_{jm} Q_{km} Q_{kl} A_{ln} A_{in}\\ &= 2(Q Q^T Q B \mathop{\rm tr} (A A^T) + Q Q^T Q A A^T)_{ji} \end{align}$$ with $(B)_{ij}= 1$ the constant unit matrix.

share|cite|improve this answer
You inexplicably dropped $\delta_{im}$ in your expansion of $\partial_{(q_i)_j} X_{mn}$. That result should read $(\delta_{im} Q_{jl} A_{ln} + Q_{jm} A_{in})$. Which changes the conclusion to $2(A^T Q^T Q A^T + A Q^T Q A) Q^T$. – lynne Feb 18 '14 at 4:03

Let's a function $F:M_{N\times N}(\mathbb{R})\to \mathbb{R}$ definid by $F(Q)=\mathrm{tr}\Big(Q^TQAQ^TQA\Big)$ then $$ \frac{\partial}{\partial q_i}\mathrm{tr}\Big(Q^TQAQ^TQA\Big)=\mathcal{D} F(Q)\cdot[0\ldots q_i\ldots 0]. $$ Here $$ [0\ldots q_i\ldots 0] = \begin{pmatrix} 0&\dots &q_{1i}&\dots & 0 \\ \vdots & \cdots & \vdots & \cdots & \vdots \\ 0&\dots &q_{ii}&\dots &0 \\ \vdots & \cdots & \vdots & \cdots & \vdots \\ 0&\dots &q_{Ni}&\dots &0 \\ \end{pmatrix} \mbox{ and } q_i= \begin{pmatrix} q_{1i} \\ \vdots \\ q_{ii} \\ \vdots \\ q_{Ni} \end{pmatrix} $$ and $\mathcal{D}F(Q_0): M_{N\times N}(\mathbb{R})\to \mathbb{R}$ is the total derivative of $F$ at $Q_0$, i.e. $$ F(Q_0+V)=F(Q_0)+\mathcal{D}F(Q_0)\cdot V+ \|V\|\cdot\rho(V),\quad \lim_{V\to 0}\frac{\rho(V)}{\|V\|}=0\quad \mbox{ and }\|V\|=\sqrt{tr(V^TV)} $$ Note that \begin{align} F(Q+V) = & tr\Big([Q+V][Q+V]^TA[Q+V]^T[Q+V]A\Big) \\ = & tr(QQ^TAQ^TQA)+ \\ & \\ & +tr(VQ^TAQ^TQA)+tr(QV^TAQ^TQA)+ \\ & +tr(QQ^TAV^TQA)+tr(QQ^TAQ^TVA)+ \\ & \\ & +tr(VV^TAQ^TQA)+ \\ & +tr(VQ^TAV^TQA)+tr(VQ^TAQ^TVA)+ \\ & +tr(QV^TAV^TQA)+tr(QV^TAQ^TVA)+ \\ & +tr(QQ^TAV^TVA)+ \\ & \\ & +tr(QV^TAV^TVA)+tr(VQ^TAV^TVA)+ \\ & +tr(VV^TAQ^TVA)+tr(VV^TAV^TQA)+ \\ & \\ & +tr(VV^TAV^TVA) \end{align} implies \begin{align} \mathcal{D}F(Q)\cdot V= & tr(VQ^TAQ^TQA)+tr(QV^TAQ^TQA) \\ & +tr(QQ^TAV^TQA)+tr(QQ^TAQ^TVA). \end{align} Play the matrix $V$ by matrix$[0,\ldots,q_i,\ldots,0]$ and calculate the answer. Good Look.

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.