Matrix Differentiation of $x^TAx$ I know the matrix differentiation of $x^TAx$ is if A is symmetric is 2Ax. I saw that some people write as $2x^TA$. Are these two results the same?
I am not sure how they are same. Can anyone explain to me..
 A: The function $f:\mathbb{R}^{n}\to\mathbb{R}$ defined by $f(x)=x^{T}Ax$ should have it's Jacobian matrix, $f'(x)$, be of size $1\times n$. So the appropriate form of the Jacobian matrix should be $2x^{T}A$. They are not equal as matrices in general (one is a row vector and one is a column vector). However, $(2x^{T}A)^{T}=2A^{T}(x^{T})^{T}=2Ax$ since $A$ is symmetric. The last sentence shows that they differ only by a matrix transposition. It may be that the person who wrote that was using the convention that $\nabla f(x)=f'(x)^{T}$ in which case they were referring to the $\nabla f(x)$ instead of the Jacobian matrix.
A: Let $\phi_A$ denote the map taking $x \to x^TAx$, so that $\phi_A(x) = x^TAx$.  Just for the record, let us suppose $x \in V$ for some vector space $V$, with field of scalars
$\Bbb F$, where $\Bbb F = \Bbb R$ or $\Bbb F = \Bbb C$, the real or complex numbers.  Then, for the sake of definiteness, we can say $\phi_A: V \to \Bbb F$.  Now if we look at $\phi_A(x + h)$ we see that
$\phi_A(x + h) = (x + h)^TA(x + h) = x^TAx + h^TAx + x^TAh + h^TAh, \tag{1}$
or
$\phi_A(x + h) = \phi_A(x) + h^TAx + x^TAh + \Vert h \Vert o(h), \tag{2}$
where $o(h)$ is a continuous function such that $\lim_{h \to 0} o(h) = 0$.  The derivative $D_x\phi_A$ of $\phi_A$ at $x$ is thus the linear functional $D_x\phi_A:V \to \Bbb F$ given by the formula
$D_x\phi_A(h) = h^TAx + x^TAh, \tag{3}$
which holds for all $A$, symmetric or not.  Now since  $h^TAx \in \Bbb F$ is a scalar it is "automatically" symmetric; it is equal to its own transpose:  $h^TAx = (h^TAx)^T = x^TA^Th^{TT} = x^TA^Th$, whence
$D_x\phi_A(h) = x^TA^Th + x^TAh, \tag{4}$
and a similar maneuver may be applied to  $x^TAh$: $x^TAh = (x^TAh)^T = h^TA^Tx^{TT} = h^TA^Tx$, yielding an alternative form of $D_x\phi_A(h)$:
$D_x\phi_A(h) = h^TAx + h^TA^Tx, \tag{5}$ 
(4) and (5) are about as far as you can go without additional assumptions on $A$; in the event $A$ is symmetric, i.e. $A = A^T$, (4) and (5) become
$D_x\phi_A(h) = 2x^TAh, \tag{6}$
and
$D_x\phi_A(h) = 2h^TAx, \tag{7}$
respectively.  I believe this shows your two formula are about as "the same" as they can get, modulo the exact choice of which vector to transpose; but you've got to do it someplace!
Hope this helps.  Cheers.
A: The gradient of a quadratic form $x^T A x$ is given by
$\nabla x^T A x = (A^T + A)x$
which is valid for arbitrary (non-symmetric) $A$. If $A$ is symmetric of course
$2 A x$ is obtained. If in this case ($A$ being symmetric) the gradient is written as row vector the other form $2 x^T A$ is obtained.
A: If you view the elements of $\mathbb{R}^n$ as column vectors, then the derivative at $x_0$ of the function $$f : \mathbb{R}^n \to \mathbb{R}, \quad x \mapsto x^T A x,$$ is the linear functional $$Df(x_0) : \mathbb{R}^n \to \mathbb{R}, \quad h \mapsto x_0^T(A+A^T)h,$$
which is represented by the $1 \times n$ matrix (i.e., row vector)
$$
 x_0^T(A+A^T).
$$
On the other hand, by definition, the gradient at $x_0$ of $f$ is the column vector $\nabla f(x_0)$ such that for all $h \in \mathbb{R}^n$,
$$
 Df(x_0)h = (\nabla f(x_0))^T h,
$$
so that
$$
 (\nabla f(x_0))^T h = Df(x_0)h = x_0^T(A+A^T)h = ((A+A^T)x_0)^T h
$$ 
for all $h$, and hence
$$
 \nabla f(x_0) = (A+A^T)x_0.
$$
