Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

What is the gradient of $\frac{1}{2}(Ax)\cdot x$ if $A$ is a nonzero symmetric $3\times 3$ matrix?

Apparently the answer is simply $Ax$. Does anyone know why?

share|improve this question
    
Assume $x$ is a column vector, otherwise it doesn't make sense to multiply matrix to a row vector due to the dimensions mismatch. Then $Ax$ is also a column vector, and again multiplying column vector to a column vector is undefined. You have two options: 1) Make it $\frac 12 (Ax)^T \cdot x$ 2) or $\frac 12 (Ax) \cdot x^T$ In first case you'll get a number, so gradient of it will be a row vector. In second case, the product is a matrix, and as far as I remember its gradient is even bigger of a matrix. In any case, you don't get $Ax$. –  Kaster Apr 18 '13 at 4:54
add comment

3 Answers

up vote 2 down vote accepted

Claim:

Let $A$ be a $k x k$ square matrix, $x$ be a $k x 1$ column vector, and let $\alpha = (1/2) x ^TAx$ be the quadratic form associated with $A$ and $x$.

Find the gradient of $\displaystyle \alpha = \frac{\partial \alpha}{\partial x} \left(\frac{1}{2} \right) x ^TAx = \frac{1}{2}\left(A + A^T\right)x$.

Furthermore, if $A$ is symmetric, this derivative reduces to $\displaystyle \left(\frac{1}{2}\right) 2 A \cdot x = A \cdot x.$

Why is that last statement true (we were told something about nonzero symmetric matrices)?

share|improve this answer
    
Nice! +1 for good work and for effort! –  amWhy Apr 18 '13 at 5:12
add comment

Yes, and it is not specific to the 3 by 3 case, it is any size square symmetric. So, could you please, please, please, actually write out the 1 by 1 case (where $A$ is just a number) and the 2 by 2 case, and write out the value of your expression and find the gradient in these two easy cases.

share|improve this answer
1  
+1 Good to see you 'round! And I love your comments & answers (and wit!) –  amWhy Apr 18 '13 at 5:13
add comment

Here is another way to see it:

Let $\phi(x) = \frac{1}{2} \langle Ax, x \rangle$.

Then \begin{eqnarray} \phi(x+\delta) &=& \frac{1}{2} \langle A(x+\delta), x+\delta \rangle \\ &=& \frac{1}{2} \left( \langle Ax,x \rangle + \langle Ax,\delta \rangle + \langle A \delta,x \rangle + \langle A\delta, \delta \rangle \right) \\ &=& \phi(x) + \frac{1}{2} \left( \langle Ax,\delta \rangle + \langle \delta,A^T x \rangle + \langle A\delta, \delta \rangle \right) \\ &=& \phi(x) + \frac{1}{2} \left( \langle Ax,\delta \rangle + \langle A^T x, \delta \rangle + \langle A\delta, \delta \rangle \right) \\ &=& \phi(x) + \langle \frac{1}{2} (A+A^T)x,\delta \rangle + \langle A\delta, \delta \rangle \end{eqnarray} It follows from this (since $|\langle A\delta, \delta \rangle| \le \|A\| \|\delta \|^2$) that $\nabla \phi(x) = \frac{1}{2} (A+A^T)x$.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.