Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

For the Quadratic Form $X^TAX; X\in\mathbb{R}^n, A\in\mathbb{R}^{n \times n}$ (which simplifies to $\Sigma_{i=0}^n\Sigma_{j=0}^nA_{ij}x_ix_j$), I tried to take the derivative wrt. X ($\Delta_X X^TAX$) and ended up with the following:

The $k^{th}$ element of the derivative represented as

$\Delta_{X_k}X^TAX=[\Sigma_{i=1}^n(A_{ik}x_k+A_{ki})x_i] + A_{kk}x_k(1-x_k)$

Does this result look right? Is there an alternative form?

I'm trying to get to the $\mu_0$ of Gaussian Discriminant Analysis by maximizing the log likelihood and I need to take the derivative of a Quadratic form. Either the result I mentioned above is wrong (shouldn't be because I went over my arithmetic several times) or the form I arrived at above is not the terribly useful to my problem (because I'm unable to proceed).

I can give more details about the problem or the steps I put down to arrive at the above result, but I didn't want to clutter to start off. Please let me know if more details are necessary.

Any link to related material is also much appreciated.

share|cite|improve this question

Let $Q(x) = x^T A x$. Then expanding $Q(x+h)-Q(x)$ and dropping the higher order term, we get $DQ(x)(h) = x^TAh+h^TAx = x^TAh+x^TA^Th = x^T(A+A^T)h$, or more typically, $\frac{\partial Q(x)}{\partial x} = x^T(A+A^T)$.

Notice that the derivative with respect to a column vector is a row vector!

share|cite|improve this answer
Could you comment on the difference expansion, please? – user191919 Feb 6 '14 at 21:00
What do you mean? Just compute $Q(x+h)-Q(x)$ explicitly. The only term missing above is $h^T A h$, and we have $|h^T A h| \le \|A\| \|h \|^2$, so the term is $O(\|h\|^2)$. – copper.hat Feb 6 '14 at 21:15
Can't see how $(x+h)^T A (x+h)$ would be obvious; I was hoping avoid opening the matrix. Any hint? – user191919 Feb 7 '14 at 2:32
I don't see how I can expand $(x+h)^T A (x+h)$ so trivially. I mean literally, why $(x+h)^T A (x+h) = x^T A x + h^TAx+x^TAh + h^T A h$ and how can you see that so quickly. It just looks a messy summation for me. – user191919 Feb 7 '14 at 10:26
There is no need to explicitly compute the sums. Matrix multiplication is associative and distributive, so we can treat them like 'numbers' in this regard. We have $A(x+h) = Ax + Ah$, $(x+h)^TA = (x^T +h^T) A = x^TA + h^T A$, etc. – copper.hat Feb 7 '14 at 16:22

$f(x) = 0.5x^\top Ax \Rightarrow Df(x) = Ax $

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.