Gradient of $x^{T}Ax$ I just came across the following
$$\nabla x^TAx = 2Ax$$
which seems like as good of a guess as any, but it certainly wasn't discussed in either my linear algebra class or my multivariable calculus class. Is there any intuitive way to see why this should be true?
 A: Try it first with a simple case, such as $\nabla x^Tx$. Well, if we have 
$$ x=(x_1,\ldots,x_n), $$then 
$$ x^Tx=\sum_{i=1}^n x_i^2. $$ It follows 
$$\nabla x^Tx = 2(x_1,\ldots,x_n)=2x $$
Now, how would you write $x^TAx$? 
Edit: If $A$ is not symmetric, you can do something similar and derive 
$$\nabla x^TAx=(A+A^T)x$$
A: It's only true if $A$ is symmetric. And as for intuition, consider the one-dimensional case: the derivative of $ax^2$ is $2ax$. I always recommend to write out the quadratic form and calculate the derivative by hand. Once you've done that, you'll understand and you'll never forget it anymore.
A: I don't know of any intuitive method. But I would like to give you the following generalization:
$$\nabla x^TAx = (A+A^T)x$$
Basically, work with $$f(x) = \sum_{i=1}^n\sum_{j=1}^nx_i a_{ij}x_j$$
and use $a_{ij}= a_{ji}$
A: This is true only if $A$ is symmetric.
For example, if $A^T=-A$, then $x^TAx=0$, and hence $\nabla x^TAx=0\ne 2Ax$.
A way to see this is
$$
(x+h)^TA(x+h)-x^TAx=x^TAh+h^TAx+{\mathcal O}(|h|^2)=2(Ax,h)+{\mathcal O}(|h|^2).
$$
A: Let's write the $i$th component. Here, I use Einstein's convention, where a sum is assumed for repeated indices. What follows assumes that $A$ is constant.
$\displaystyle \left[ \vec{\nabla}\left( x^T A x \right)\right]_i = \partial_i \left( x_j A_{jk} x_k\right) = \left(\partial_i x_j\right) A_{jk} x_k + x_j A_{jk} \left( \partial_i x_k\right) = \delta_{ij} A_{jk} x_k + x_j A_{jk} \delta_{ik} = A_{ik} x_k + x_j A_{ji}.$
Therefore,
$\displaystyle \vec{\nabla}\left( x^T A x \right) = A x + x^T A$.
If $A$ is symmetric, then
$\displaystyle A x + x^T A = 2 A x$.
A: I assume A is quadratic(otherwise the term in your question does not hold)
If you know that:


*

*any symmetric matrix $A$ can be decomposed to $A=LU$

*$\nabla Ax=A$ [and $\nabla x^TA = \nabla(A^Tx)^T=(\nabla A^Tx)^T=(A^T)^T=A$]

*The product rule holds: $\nabla uv=(\nabla u)v+u\nabla v$
You can write: 
$$\nabla x^TAx \\ 
=\nabla x^TLUx \\ 
=(\nabla x^TL) Ux + x^TL \nabla Ux \\
=LUx+x^TLU \\
=Ax + x^TA \\
=Ax + x^TA^T \\
=Ax + Ax\\
=2Ax \\
$$
