# Matrix Differentiation of $x^TAx$ [duplicate]

I know the matrix differentiation of $x^TAx$ is if A is symmetric is 2Ax. I saw that some people write as $2x^TA$. Are these two results the same?

I am not sure how they are same. Can anyone explain to me..

• You should clarify that you want the matrix derivative wrt $x$ Commented Apr 24, 2020 at 22:57

The function $f:\mathbb{R}^{n}\to\mathbb{R}$ defined by $f(x)=x^{T}Ax$ should have it's Jacobian matrix, $f'(x)$, be of size $1\times n$. So the appropriate form of the Jacobian matrix should be $2x^{T}A$. They are not equal as matrices in general (one is a row vector and one is a column vector). However, $(2x^{T}A)^{T}=2A^{T}(x^{T})^{T}=2Ax$ since $A$ is symmetric. The last sentence shows that they differ only by a matrix transposition. It may be that the person who wrote that was using the convention that $\nabla f(x)=f'(x)^{T}$ in which case they were referring to the $\nabla f(x)$ instead of the Jacobian matrix.

• Note that, while $f'(x)$ is a $1 \times n$ matrix, some people use the convention that $\nabla f(x)$ is an $n \times 1$ column vector. So by that convention we would have $\nabla f(x) = f'(x)^T$. For example, in optimization literature $\nabla f(x)$ is usually a column vector. Commented Sep 6, 2013 at 2:43
• I was not aware of that convention. Thank you for the information littleO. Commented Sep 6, 2013 at 2:46

Let $\phi_A$ denote the map taking $x \to x^TAx$, so that $\phi_A(x) = x^TAx$. Just for the record, let us suppose $x \in V$ for some vector space $V$, with field of scalars $\Bbb F$, where $\Bbb F = \Bbb R$ or $\Bbb F = \Bbb C$, the real or complex numbers. Then, for the sake of definiteness, we can say $\phi_A: V \to \Bbb F$. Now if we look at $\phi_A(x + h)$ we see that

$\phi_A(x + h) = (x + h)^TA(x + h) = x^TAx + h^TAx + x^TAh + h^TAh, \tag{1}$

or

$\phi_A(x + h) = \phi_A(x) + h^TAx + x^TAh + \Vert h \Vert o(h), \tag{2}$

where $o(h)$ is a continuous function such that $\lim_{h \to 0} o(h) = 0$. The derivative $D_x\phi_A$ of $\phi_A$ at $x$ is thus the linear functional $D_x\phi_A:V \to \Bbb F$ given by the formula

$D_x\phi_A(h) = h^TAx + x^TAh, \tag{3}$

which holds for all $A$, symmetric or not. Now since $h^TAx \in \Bbb F$ is a scalar it is "automatically" symmetric; it is equal to its own transpose: $h^TAx = (h^TAx)^T = x^TA^Th^{TT} = x^TA^Th$, whence

$D_x\phi_A(h) = x^TA^Th + x^TAh, \tag{4}$

and a similar maneuver may be applied to $x^TAh$: $x^TAh = (x^TAh)^T = h^TA^Tx^{TT} = h^TA^Tx$, yielding an alternative form of $D_x\phi_A(h)$:

$D_x\phi_A(h) = h^TAx + h^TA^Tx, \tag{5}$

(4) and (5) are about as far as you can go without additional assumptions on $A$; in the event $A$ is symmetric, i.e. $A = A^T$, (4) and (5) become

$D_x\phi_A(h) = 2x^TAh, \tag{6}$

and

$D_x\phi_A(h) = 2h^TAx, \tag{7}$

respectively. I believe this shows your two formula are about as "the same" as they can get, modulo the exact choice of which vector to transpose; but you've got to do it someplace!

Hope this helps. Cheers.

The gradient of a quadratic form $$x^T A x$$ is given by

$$\nabla x^T A x = (A^T + A)x$$

which is valid for arbitrary (non-symmetric) $$A$$. If $$A$$ is symmetric of course $$2 A x$$ is obtained. If in this case ($$A$$ being symmetric) the gradient is written as row vector the other form $$2 x^T A$$ is obtained.

If you view the elements of $\mathbb{R}^n$ as column vectors, then the derivative at $x_0$ of the function $$f : \mathbb{R}^n \to \mathbb{R}, \quad x \mapsto x^T A x,$$ is the linear functional $$Df(x_0) : \mathbb{R}^n \to \mathbb{R}, \quad h \mapsto x_0^T(A+A^T)h,$$ which is represented by the $1 \times n$ matrix (i.e., row vector) $$x_0^T(A+A^T).$$ On the other hand, by definition, the gradient at $x_0$ of $f$ is the column vector $\nabla f(x_0)$ such that for all $h \in \mathbb{R}^n$, $$Df(x_0)h = (\nabla f(x_0))^T h,$$ so that $$(\nabla f(x_0))^T h = Df(x_0)h = x_0^T(A+A^T)h = ((A+A^T)x_0)^T h$$ for all $h$, and hence $$\nabla f(x_0) = (A+A^T)x_0.$$