For the Quadratic Form $X^TAX; X\in\mathbb{R}^n, A\in\mathbb{R}^{n \times n}$ (which simplifies to $\Sigma_{i=0}^n\Sigma_{j=0}^nA_{ij}x_ix_j$), I tried to take the derivative wrt. X ($\Delta_X X^TAX$) and ended up with the following:

The $k^{th}$ element of the derivative represented as

$\Delta_{X_k}X^TAX=[\Sigma_{i=1}^n(A_{ik}x_k+A_{ki})x_i] + A_{kk}x_k(1-x_k)$

Does this result look right? Is there an alternative form?

I'm trying to get to the $\mu_0$ of Gaussian Discriminant Analysis by maximizing the log likelihood and I need to take the derivative of a Quadratic form. Either the result I mentioned above is wrong (shouldn't be because I went over my arithmetic several times) or the form I arrived at above is not the terribly useful to my problem (because I'm unable to proceed).

I can give more details about the problem or the steps I put down to arrive at the above result, but I didn't want to clutter to start off. Please let me know if more details are necessary.

Any link to related material is also much appreciated.

-

Let $Q(x) = x^T A x$. Then expanding $Q(x+h)-Q(x)$ and dropping the higher order term, we get $DQ(x)(h) = h^TAx+x^TAh = x^T(A+A^T)h$, or more typically, $\frac{\partial Q(x)}{\partial x} = x^T(A+A^T)$.
What do you mean? Just compute $Q(x+h)-Q(x)$ explicitly. The only term missing above is $h^T A h$, and we have $|h^T A h| \le \|A\| \|h \|^2$, so the term is $O(\|h\|^2)$. –  copper.hat Feb 6 '14 at 21:15
Can't see how $(x+h)^T A (x+h)$ would be obvious; I was hoping avoid opening the matrix. Any hint? –  user191919 Feb 7 '14 at 2:32
I don't see how I can expand $(x+h)^T A (x+h)$ so trivially. I mean literally, why $(x+h)^T A (x+h) = x^T A x + h^TAx+x^TAh + h^T A h$ and how can you see that so quickly. It just looks a messy summation for me. –  user191919 Feb 7 '14 at 10:26
There is no need to explicitly compute the sums. Matrix multiplication is associative and distributive, so we can treat them like 'numbers' in this regard. We have $A(x+h) = Ax + Ah$, $(x+h)^TA = (x^T +h^T) A = x^TA + h^T A$, etc. –  copper.hat Feb 7 '14 at 16:22
$f(x) = 0.5x^\top Ax \Rightarrow Df(x) = Ax$