$$\frac{\partial a^TX a}{\partial X} = \frac{\partial a^TX^T a}{\partial X} = aa^T \tag 1$$

I got (1) from the Matrix Cookbook. But I don't see how you derive it? Why isn't it $a^Ta$.

Assume that $a$ is an arbitrary vector of real numbers of length $n$ and $X$ is an arbitrary $n\times n$ matrix.

  • $\begingroup$ I don't understand what is a partial derivative w.r.t. a matrix. $\endgroup$
    – Bernard
    Aug 20 '15 at 22:41
  • $\begingroup$ @Bernard : Maybe you should make that a question. ${}\qquad{}$ $\endgroup$ Aug 21 '15 at 3:24

Write out $a^TXa$ by component:

$$ a^TXa = \sum_{ij} a_ix_{ij}a_j $$

Realize that $\frac{\partial}{\partial X}$ is shorthand for the matrix composed of entries $\frac{\partial}{\partial x_{ij}}$. Therefore,

$$ \frac{\partial a^TXa}{\partial x_{ij}} = a_ia_j $$

Stack these into a matrix, and this is exactly $aa^T$. Why not $a^Ta$? Just look at the dimensions! :)


A more general result is as follows. Let $M:=\text{Mat}_{n\times n}(\mathbb{R})$. Define $f:M\to \mathbb{R}$ to be $f(X)=\text{trace}(AX)$ for some fixed $A \in M$. Then, $$\frac{\partial}{\partial X}\,f(X)=A^\top\,.$$ In this particular case, $A=aa^\top$, so $f(X)=\text{trace}\left(aa^\top X\right)=\text{trace}\left(a^\top Xa\right)=a^\top Xa$. Hence, $$\frac{\partial}{\partial X}\,\left(a^\top X a\right)=\left(aa^\top\right)^\top =aa^\top\,.$$

  • 1
    $\begingroup$ there is a typo at the end $a^\top a$ instead of $aa^\top$ $\endgroup$
    – user251257
    Aug 21 '15 at 2:00
  • $\begingroup$ @user251257. Thanks, fixed. $\endgroup$ Aug 21 '15 at 2:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.