# Why is $\nabla_x x^T A x = 2 A x$?

Let $f: \mathbb{R}^n \rightarrow \mathbb{R}$ and $f(x) = x^T A x$. I will denote $\nabla_x$ or $\nabla$ as the gradient to some vector-valued variable and $\nabla^2$ or $H$ as the Hessian.

The lecturer postulated that $\nabla f(x) = 2 A x$, and that $\nabla^2 f(x) = 2A$.

It's not immediately clear to me that this is true. What I thought of, was that $\nabla f(x)$ always yields a column vector, and that therefore $\nabla f(x) = 2 A x$. But this feels more like a trick (to remember it) and not like a proof to me.

How does one derive $\nabla_x x^T A x = 2 A x$? Why can't it be $2 (x^T A)^T = A^T x$?

• Not in general, but $A$ only occurs in the quadratic form and nowhere else. Therefore the antisymmetric part cancels (right?), so you can take $B = \frac{1}{2}(A+A^T)$. However, if we leave $A$ as it is, it's $Ax \neq (x^T A)^T$ if it's not symmetric which confuses me. Oct 23, 2016 at 12:35
• Use on the web the keyword "gradient". Asking for "gradient of X^TAX", I had at once many answers, like (math.stackexchange.com/q/482742) Oct 23, 2016 at 12:54
• For an in-depth understanding of matricial (and graphical) understanding ofquadratic forms, I advise you this document (www2.econ.iastate.edu/classes/econ501/Hallam/documents/…). Oct 23, 2016 at 13:02

You need $A$ to be symmetric for that.

$x^TAx=\langle Ax, x\rangle=:f(x).$

$f(x+h)=\langle A(x+h), x+h \rangle=\langle Ax, x \rangle + \langle Ax,h \rangle +\langle x,Ah \rangle+ \langle Ah,h \rangle.$

Therefore, the derivative is given by $f'_x=\langle Ax, \cdot \rangle +\langle x, A \cdot \rangle.$

Because $A$ is symmetric, (*) $$f'_x=\langle 2Ax, \cdot \rangle.$$

Since the definition of $\nabla_x f$ is the vector such that $f'_x= \langle \nabla_x f, \cdot \rangle$ (which exists and is unique by Riesz), we get $$\nabla_x f= 2Ax.$$

(*) Note that $f'_x=\langle Ax, \cdot \rangle +\langle x, A \cdot \rangle=\langle (A+A^T)x, \cdot\rangle .$ Therefore, $f'_x=\langle 2Ax, \cdot \rangle$ if and only if $2A=A+A^T$, which occurs if and only if $A$ is symmetric.

• So symmetry of $A$ is an absolutely necessary condition for $\nabla_x f = 2Ax$? Oct 23, 2016 at 12:43
• @Jasper Yes. See the edit. Oct 23, 2016 at 12:48
• > Therefore, the derivative is given by 𝑓′𝑥=⟨𝐴𝑥,⋅⟩+⟨𝑥,𝐴⋅⟩. can someone explain how this was found? Dec 23, 2021 at 0:28