How to find $\frac{\partial x^TAx}{\partial A}$ $y = x^TAx$. I want to find the $\frac{\partial x^TAx}{\partial A}$; that is, $\frac{\partial y}{\partial A}$.
I know the answer is $\frac{\partial y}{\partial A} = xx^T$. I'm not totally clear on how to get that, and I'm really not clear as to why the $x$'s are now $xx^T$. In other words, why is $x^Tx$ not correct?
Note: I take $x$ = to be a column vector.
 A: The differential of your map $f=x^TAx$ is $df=x^TdAx$. It is a linear map sending $H\mapsto x^T H x$, a scalar. Then write $tr(x^T H x)=tr(xx^T H)=\langle\nabla f,H\rangle_F$ with the "gradient vector" $\nabla f=(xx^T)^T=xx^T.$
A: Considering a more general problem may shed light on the current problem. 
Let $$y = x^TAw$$
where $A$ is rectangular, i.e. $\,\,x$ and $w\,$ have different lengths.
Write the function in terms of the Frobenius product and find its differential and derivative as
$$\eqalign{
  y &= xw^T:A  \cr\cr
 dy &= xw^T:dA \cr\cr
 \frac{\partial y}{\partial A} &= xw^T \cr
}$$
Your problem is a special case of the above, where $\,w=x$.
A: Let's use Einstein Notation
$y = x_i A_{ij} x_j$

$\frac{\partial A_{ij}}{\partial A_{mn}} = \delta_{im}\delta_{nj}$ (4th order tensor)
where $\delta$ is the Kronecker Delta

$\frac{\partial y}{\partial A_{mn}} = \frac{\partial (x_i A_{ij} x_j)}{\partial A_{mn}} = x_i \frac{\partial A_{ij}}{\partial A_{mn}}x_j = x_i\delta_{im}\delta_{nj}x_j = x_m x_n$
Back in matricial notation: $\boxed{\frac{\partial y}{\partial A} = x\,x^T}$
