Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

What is the gradient of $\frac{1}{2}(Ax)\cdot x$ if $A$ is a nonzero symmetric $3\times 3$ matrix?

Apparently the answer is simply $Ax$. Does anyone know why?

share|cite|improve this question
Assume $x$ is a column vector, otherwise it doesn't make sense to multiply matrix to a row vector due to the dimensions mismatch. Then $Ax$ is also a column vector, and again multiplying column vector to a column vector is undefined. You have two options: 1) Make it $\frac 12 (Ax)^T \cdot x$ 2) or $\frac 12 (Ax) \cdot x^T$ In first case you'll get a number, so gradient of it will be a row vector. In second case, the product is a matrix, and as far as I remember its gradient is even bigger of a matrix. In any case, you don't get $Ax$. – Kaster Apr 18 '13 at 4:54
up vote 2 down vote accepted


Let $A$ be a $k x k$ square matrix, $x$ be a $k x 1$ column vector, and let $\alpha = (1/2) x ^TAx$ be the quadratic form associated with $A$ and $x$.

Find the gradient of $\displaystyle \alpha = \frac{\partial \alpha}{\partial x} \left(\frac{1}{2} \right) x ^TAx = \frac{1}{2}\left(A + A^T\right)x$.

Furthermore, if $A$ is symmetric, this derivative reduces to $\displaystyle \left(\frac{1}{2}\right) 2 A \cdot x = A \cdot x.$

Why is that last statement true (we were told something about nonzero symmetric matrices)?

share|cite|improve this answer
Nice! +1 for good work and for effort! – amWhy Apr 18 '13 at 5:12

Yes, and it is not specific to the 3 by 3 case, it is any size square symmetric. So, could you please, please, please, actually write out the 1 by 1 case (where $A$ is just a number) and the 2 by 2 case, and write out the value of your expression and find the gradient in these two easy cases.

share|cite|improve this answer
+1 Good to see you 'round! And I love your comments & answers (and wit!) – amWhy Apr 18 '13 at 5:13

Here is another way to see it:

Let $\phi(x) = \frac{1}{2} \langle Ax, x \rangle$.

Then \begin{eqnarray} \phi(x+\delta) &=& \frac{1}{2} \langle A(x+\delta), x+\delta \rangle \\ &=& \frac{1}{2} \left( \langle Ax,x \rangle + \langle Ax,\delta \rangle + \langle A \delta,x \rangle + \langle A\delta, \delta \rangle \right) \\ &=& \phi(x) + \frac{1}{2} \left( \langle Ax,\delta \rangle + \langle \delta,A^T x \rangle + \langle A\delta, \delta \rangle \right) \\ &=& \phi(x) + \frac{1}{2} \left( \langle Ax,\delta \rangle + \langle A^T x, \delta \rangle + \langle A\delta, \delta \rangle \right) \\ &=& \phi(x) + \langle \frac{1}{2} (A+A^T)x,\delta \rangle + \langle A\delta, \delta \rangle \end{eqnarray} It follows from this (since $|\langle A\delta, \delta \rangle| \le \|A\| \|\delta \|^2$) that $\nabla \phi(x) = \frac{1}{2} (A+A^T)x$.

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.