Someone asks a different part of this question here. I wonder how can we derive from (46) to (47)? Especially how to think the way to get the transpose?

  • $\begingroup$ What is the $k$th entry on the right-hand side of (47)? It is the right-hand side of (46), no? $\endgroup$ Sep 10, 2017 at 21:17
  • $\begingroup$ You are right, but I think it's easy to derive from (47) to (46), but harder to think why (46) can go to (47), and have difficulty coming with the idea $\endgroup$
    – user21
    Sep 10, 2017 at 22:20

1 Answer 1


I think the confusion might be coming from the row/column conventions for matrix calculus, which are often not well-explained in courses.

The important thing to remember is that matrix-vector multiplication can be defined per-component by $$ [Ax]_{i} = \sum_j a_{ij} x_j $$ And, for the transpose, we get $$ [A^Tx]_{i} = \sum_j a_{ji} x_j $$ So, (46) can be rewritten as: $$ \frac{\partial \alpha}{\partial x_k} = \sum_j a_{kj}x_j + \sum_i a_{ik}x_i = [Ax]_k + [A^Tx]_k $$ Clearly, the quantity $\frac{\partial \alpha}{\partial x}$ should be a vector with components $\frac{\partial \alpha}{\partial x_k}$. Now, we are looking at the Jacobian of $\alpha(x)=\sum_{i,j} a_{ij}x_ix_j$, so it should be a row vector. However, we have written the above as components of column vectors. So it is better to write: $$ \frac{\partial \alpha}{\partial x_k} = \sum_j a_{kj}x_j + \sum_i a_{ik}x_i = [x^TA^T]_k + [x^TA]_k $$ Note this makes no difference to the components since a transposed vector has the same components for a given index. However, now we correctly get a row vector when we generalize this as a matrix derivative: $$ \frac{\partial \alpha}{\partial x} = x^TA^T + x^TA = x^T(A^T+A) $$ by simply removing the component index.

Hopefully this way of looking at it helps!


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .