'What is $\partial [x^\top x]/\partial x$' debate For $x \in \mathbb R^n$ , is $\frac{\partial[x^Tx]}{\partial[x]}$ equal to  $2x$ or $2x^T$? There are a lot of discrepancies about this, such as Vector derivation of $x^Tx$.
Another example of the discrepancy is the following, Computing matrix-vector calculus derivatives, where Jonas claimed that Par's answer's 4th line is incorrect. As a result, Par's answer was 'a' and Jonas' answer was 'a' transpose. Which answerer to the question of "what's the derivative of x^T a with respect to x" was correct?
 A: This is a perfect example of the sort of mess you can get into by working in coordinates. Let's work abstractly: $V$ is a finite-dimensional real inner product space, with inner product $\langle -, - \rangle$. We'd like to know the derivative of the quadratic form $q(x) = \langle x, x \rangle : V \to \mathbb{R}$ at a point $a \in V$. The most important thing to know about this derivative is its type: what sort of mathematical object is it? The answer is that it is a linear functional $V \to \mathbb{R}$; that is, it's a covector. (We can then use the inner product to identify vectors and covectors if we want to, but this just confuses the issue.)
Okay, now let's actually compute it. We have
$$q(a + dx) = \langle a + dx, a + dx \rangle = \langle a, a \rangle + 2 \langle a, dx \rangle + O(dx^2)$$
from which it follows that the derivative at $a$ is the linear functional $dx \mapsto 2 \langle a, dx \rangle$. Using the inner product to identify vectors and covectors, the corresponding vector is $2a$. 
A: I don't like @Qiaochu's answer since it isn't exactly enlightening as to what the problem is.
I think the problem here stems from not defining what precisely $\partial \vec{f}/\partial \vec{x}$ means.
If you you assume $\vec{x}$ and $\vec{f}$ are column vectors and you use this Jacobian matrix definition $$\frac{\partial \vec{f}}{\partial \vec{x}} = \begin{bmatrix}\vdots \\ \partial f_i/\partial \vec{x} \\ \vdots\end{bmatrix} = \begin{bmatrix}\vdots \\ \vec{\nabla} f_i \\ \vdots\end{bmatrix}$$ then you see $\partial f_i/\partial \vec{x} = \vec{\nabla} f_i$ must be a row vector, not a column vector. So you get $$\frac{\partial}{\partial \vec{x}} \vec{x}^\top \vec{x} = 2 \vec{x}^\top$$ whereas if you define the Jacobian to be the transpose then you get back a column vector, $2\vec{x}$.
