It's stated that the gradient of:

$$\frac{1}{2}x^TAx - b^Tx +c$$

is

$$\frac{1}{2}A^Tx + \frac{1}{2}Ax - b$$

How do you grind out this equation? Or specifically, how do you get from $x^TAx$ to $A^Tx + Ax$?

-

The only thing you need to remember/know is that $$\dfrac{\partial (x^Ty)}{\partial x} = y$$ and the chain rule, which goes as $$\dfrac{d(f(x,y))}{d x} = \dfrac{\partial (f(x,y))}{\partial x} + \dfrac{\partial y^T}{\partial x} \dfrac{\partial (f(x,y))}{\partial y}$$ Hence, $$\dfrac{\partial (b^Tx)}{\partial x} = \dfrac{\partial (x^Tb)}{\partial x} = b$$
$$\dfrac{\partial (x^TAx)}{\partial x} = \dfrac{\partial (x^Ty)}{\partial x} + \dfrac{\partial y^T}{\partial x} \dfrac{\partial (x^Ty)}{\partial y}$$ where $y = Ax$.
$$\dfrac{\partial (x^TAx)}{\partial x} = \dfrac{\partial (x^Ty)}{\partial x} + \dfrac{\partial y^T}{\partial x} \dfrac{\partial (x^Ty)}{\partial y} = y + \dfrac{\partial (x^TA^T)}{\partial x} \dfrac{\partial (x^Ty)}{\partial y} = y + A^Tx = (A+A^T)x$$
To help future generations: the full specification of the chain rule used here is $$\frac{df(g,h)}{dx} = \frac{d(g(x)^T)}{dx} \frac{\partial f(g,h)}{\partial g} + \frac{d(h(x)^T)}{dx} \frac{\partial f(g,h)}{\partial h}$$ The order of multiplication is very important since we're dealing with vectors! – Neil Traft Sep 23 '14 at 9:58