# Matrix Differentiation proof II

Someone asks a different part of this question here. I wonder how can we derive from (46) to (47)? Especially how to think the way to get the transpose?

• What is the $k$th entry on the right-hand side of (47)? It is the right-hand side of (46), no? Sep 10, 2017 at 21:17
• You are right, but I think it's easy to derive from (47) to (46), but harder to think why (46) can go to (47), and have difficulty coming with the idea Sep 10, 2017 at 22:20

The important thing to remember is that matrix-vector multiplication can be defined per-component by $$[Ax]_{i} = \sum_j a_{ij} x_j$$ And, for the transpose, we get $$[A^Tx]_{i} = \sum_j a_{ji} x_j$$ So, (46) can be rewritten as: $$\frac{\partial \alpha}{\partial x_k} = \sum_j a_{kj}x_j + \sum_i a_{ik}x_i = [Ax]_k + [A^Tx]_k$$ Clearly, the quantity $\frac{\partial \alpha}{\partial x}$ should be a vector with components $\frac{\partial \alpha}{\partial x_k}$. Now, we are looking at the Jacobian of $\alpha(x)=\sum_{i,j} a_{ij}x_ix_j$, so it should be a row vector. However, we have written the above as components of column vectors. So it is better to write: $$\frac{\partial \alpha}{\partial x_k} = \sum_j a_{kj}x_j + \sum_i a_{ik}x_i = [x^TA^T]_k + [x^TA]_k$$ Note this makes no difference to the components since a transposed vector has the same components for a given index. However, now we correctly get a row vector when we generalize this as a matrix derivative: $$\frac{\partial \alpha}{\partial x} = x^TA^T + x^TA = x^T(A^T+A)$$ by simply removing the component index.