0
$\begingroup$

How can we find: $$ \frac{\partial}{\partial\mathbf{x}}\operatorname{tr}(\mathbf{xx}^{T})=\;? $$ Now from the matrix cookbook, I found that: $$ \frac{\partial}{\partial\mathbf{X}}\operatorname{tr}(\mathbf{X})=\mathbf{I} $$ but in my case, I am deriving with respect to a vector.

$\endgroup$
2
  • $\begingroup$ en.wikipedia.org/wiki/Matrix_calculus#Scalar-by-vector tells you how to interpret the derivative. $\endgroup$
    – Arthur
    Sep 30, 2022 at 7:19
  • $\begingroup$ I suspect that $\mathbf x = (x_1,\ldots,x_n)^T$ is a column vector. Then $\operatorname{tr}(\mathbf x \mathbf x^T) = x_1^2 + \ldots + x_n^2$. You have not used enough words (or notation) to explain why you want this. $\endgroup$
    – hardmath
    Oct 8, 2022 at 1:51

1 Answer 1

5
$\begingroup$

Since $tr(xx^T) = x^Tx$, the derivative is simply $2x$ (or $2x^T$ if you are interpreting the gradient as a row vector).

$\endgroup$
3
  • $\begingroup$ "if you are interpreting the gradient as a row vector" Is there any other way? Gradients are basically meant to be dot-multiplied with vectors to give directional derivatives, for instance, which makes it very natural to write as a row vector. Also, how the gradient responds to a change of basis in the domain puts it pretty firmly in row-vector land. In more fancy jargon, assuming the domain of the function is a vector space, the gradient is a function on the dual space. $\endgroup$
    – Arthur
    Sep 30, 2022 at 8:43
  • 2
    $\begingroup$ A small remark: $trace(xx^T) = x^Tx$ is a particular case of a more general property, valid for any matrices $A \ (n \times p)$ and $B \ (p \times n)$ : $trace(AB)=trace(BA)$ $\endgroup$
    – Jean Marie
    Sep 30, 2022 at 8:53
  • $\begingroup$ @Arthur I agree that there's a case to be made for row vector gradients, but it is certainly not universally written that way. For example the gradient operator $\nabla$ is sometimes interpreted as a row vector itself (for the reasons you mentioned) so that the divergence of a column vector field $\vec u$ is $\nabla\cdot \vec u$. But that would mean vector fields like $\nabla p$ for a scalar field $p$ need to be interpreted themselves as column vectors in order to operations like this to make sense. There is some discussion of the nuances here: en.wikipedia.org/wiki/Gradient#Derivative $\endgroup$
    – Andrew
    Oct 1, 2022 at 20:39

Not the answer you're looking for? Browse other questions tagged .