Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

For the Quadratic Form $X^TAX; X\in\mathbb{R}^n, A\in\mathbb{R}^{n \times n}$ (which simplifies to $\Sigma_{i=0}^n\Sigma_{j=0}^nA_{ij}x_ix_j$), I tried to take the derivative wrt. X ($\Delta_X X^TAX$) and ended up with the following:

The $k^{th}$ element of the derivative represented as

$\Delta_{X_k}X^TAX=[\Sigma_{i=1}^n(A_{ik}x_k+A_{ki})x_i] + A_{kk}x_k(1-x_k)$

Does this result look right? Is there an alternative form?

I'm trying to get to the $\mu_0$ of Gaussian Discriminant Analysis by maximizing the log likelihood and I need to take the derivative of a Quadratic form. Either the result I mentioned above is wrong (shouldn't be because I went over my arithmetic several times) or the form I arrived at above is not the terribly useful to my problem (because I'm unable to proceed).

I can give more details about the problem or the steps I put down to arrive at the above result, but I didn't want to clutter to start off. Please let me know if more details are necessary.

Any link to related material is also much appreciated.

share|improve this question
add comment

2 Answers

Let $Q(x) = x^T A x$. Then expanding $Q(x+h)-Q(x)$ and dropping the higher order term, we get $DQ(x)(h) = h^TAx+x^TAh = x^T(A+A^T)h$, or more typically, $\frac{\partial Q(x)}{\partial x} = x^T(A+A^T)$.

share|improve this answer
    
Could you comment on the difference expansion, please? –  user191919 Feb 6 at 21:00
    
What do you mean? Just compute $Q(x+h)-Q(x)$ explicitly. The only term missing above is $h^T A h$, and we have $|h^T A h| \le \|A\| \|h \|^2$, so the term is $O(\|h\|^2)$. –  copper.hat Feb 6 at 21:15
    
Can't see how $(x+h)^T A (x+h)$ would be obvious; I was hoping avoid opening the matrix. Any hint? –  user191919 Feb 7 at 2:32
    
I still don't understand what you are asking. Computing the derivative is much like computing the derivative of $x \mapsto x^2$ from first principles. I don't understand what you mean by 'opening the matrix'. –  copper.hat Feb 7 at 3:11
    
I don't see how I can expand $(x+h)^T A (x+h)$ so trivially. I mean literally, why $(x+h)^T A (x+h) = x^T A x + h^TAx+x^TAh + h^T A h$ and how can you see that so quickly. It just looks a messy summation for me. –  user191919 Feb 7 at 10:26
show 1 more comment

$f(x) = 0.5x^\top Ax \Rightarrow Df(x) = Ax $

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.