# Gradient of a dot product with a matrix

I have a function:

$$f(\vec x) = \frac 12 \vec x \cdot (A \vec x) - \vec x \cdot \vec b$$

Now I have to find the gradient of the function: $\vec \nabla f(\vec x)$.

Is there some easy way to do this?

Currently I have this and it feels pretty cumbersome.

\begin{align*} f(\vec x) &= \frac 12 \left( x_i \left( A \vec x \right)^i \right) - x^i b_i \\ \left(\nabla f(\vec x) \right)_i &= \frac 12 \frac{\partial}{\partial x_i} \left( x_i \left( A \vec x \right)^i\right) - b_i \\ \left(\nabla f(\vec x) \right)_i &= \frac 12 \left( \left( A \vec x \right)_i + x_i \frac{\partial}{\partial x_i} \left( A \vec x \right)_i\right) - b_i \\ \left(\nabla f(\vec x) \right)_i &= \frac 12 \left( a_i^j x_j + x_i \frac{\partial}{\partial x_i} a_i^j x_j \right) - b_i \\ \left(\nabla f(\vec x) \right)_i &= \frac 12 \left( a_i^j x_j + x_i a_{ii} \right) - b_i \\ \left(\nabla f(\vec x) \right)_i &= \frac 12 \left( a_i^j x_j + a_{ii} x_i \right) - b_i \\ \nabla f(\vec x) &= \frac 12 A \vec x + \frac 12 a_{ii} x_i \vec{e^i} - \vec b \\ \end{align*}

I now use a new index $z$ for the indexing and have now $i$ free for the dot products:

$\newcommand{\vnabla}{\vec{\nabla}} \newcommand{\half}{\frac{1}{2}}$

\begin{align*} f(\vec x) &= \half \left( x_i \left( A \vec x \right)^i \right) - x^i b_i \\ \left(\nabla f(\vec x) \right)_z &= \half \frac{\partial}{\partial x_z} \left( x_i \left( A \vec x \right)^i\right) - b_z \\ \left(\vnabla f(\vec x) \right)_z &= \half \left( \left( A \vec x \right)_z + x_i \frac{\partial}{\partial x_z} \left( A \vec x \right)^i\right) - b_z \\ \left(\vnabla f(\vec x) \right)_z &= \half \left( \left( A \vec x \right)_z + x^i \frac{\partial}{\partial x_z} a_i^j x_j \right) - b_z \\ \left(\vnabla f(\vec x) \right)_z &= \half \left( \left( A \vec x \right)_z + x^i a_{iz} \right) - b_z \\ \left(\vnabla f(\vec x) \right)_z &= \half \left( \left( A \vec x \right)_z + a_{iz} x^i \right) - b_z \\ \vnabla f(\vec x) &= \half A \vec x + \half A^t \vec x - \vec b \\ \vnabla f(\vec x) &= A \vec x - \vec b \\ \end{align*}

The last step can only be done if $A$ is symmetric, of course.

-
I don't get what you mean by $(f(x))_i$ - your $f$ is scalar... I suggest to write down the difference quotient $(f(x+th) - f(x))/t$ for some vector $h$ and positive $t$ and let $t\to 0$. – Dirk May 4 '12 at 18:32
That indeed is nonsense. I fixed it. – Martin Ueding May 4 '12 at 18:47
The answer should just be $\nabla f(x) = Ax - b$. You've messed up somewhere; probably because you reused the index $i$ in $\partial/\partial x_i$ and $x_i (Ax)^i$. – Rahul May 4 '12 at 18:53
Oh yes, sure enough. Rahul has it right. Your Einstein notation in the very first step has to use some index other than $i$ or you get into trouble trying to calculate the components of $\nabla f$ in the next step. – Paul Z May 4 '12 at 18:56
Yes, my mistake, the general answer is $\nabla f(x) = A_S x - b$ where $A_S = \frac12(A + A^T)$ is the symmetric part of $A$. Generally, when you have a term $x^TAx$, you morally ought to have $A$ symmetric, because the antisymmetric part does not affect the value at all. – Rahul May 4 '12 at 20:13