# Maximize $f(\mathbf{x})={1 \over 2}(A \mathbf{x}) \cdot \mathbf{x}$

Let $A$ be a nonzero symmetric $3 \times 3$ matrix. Consider the function $f(\mathbf{x})={1 \over 2}(A \mathbf{x}) \cdot \mathbf{x}$.

What is $\nabla f$?

-
$\lim_{\epsilon\to 0}[f(x+\epsilon y)-f(x)]/\epsilon=(1/2)[Ax\cdot y+Ay\cdot x]=Ax\cdot y$ since $A$ is symmetric –  yoyo Mar 1 '13 at 1:25

$f(x)=\frac{1}{2} x^T A x$ and therefore, $\nabla f(x)=Ax$. see here

-
Could you elaborate? –  Sean Haugh Mar 1 '13 at 1:19
The formula $\frac{\partial x^T A x}{\partial x}=2Ax$ appears there explicitly, and it's true for any $n \times n$ matrix $A$. The only proof I can think of is a direct calculation, i.e. write $A=\begin{pmatrix} a_{11} & a_{12} & a_{13} \\ a_{21} & a_{22} & a_{23} \\ a_{31} & a_{32} & a_{33} \end{pmatrix}$, multiply and take partial derivatives. –  user1337 Mar 1 '13 at 1:25

Here is one way among others to do that.

Fix $x$. Then for every $h$ $$f(x+h)=\frac{(A(x+h),x+h)}{2}=f(x)+\frac{(Ax,h)+(Ah,x)}{2}+\frac{(Ah,h)}{2}.$$

Now, since $A$ is symmetric, we can simplify $$L(h)=\frac{(Ax,h)+(Ah,x)}{2}=(Ax,h).$$ The latter is linear and of course bounded. It is our candidate for being the derivative of $f$ at $x$. By definition, for this to be true, it is equivalent to show that the remainder $r(h)=f(x+h)-f(x)-L(h)$ satisfies $$\lim_{h\rightarrow 0} \frac{r(h)}{\|h\|}=0.$$

By Cauchy-Schwarz, we have $$\lvert \frac{r(h)}{\|h\|}\rvert = \lvert \frac{(Ah,h)}{2}\rvert \leq\frac{\|Ah\|\|h\|}{2\|h\|}\leq \frac{\|A\|\|h\|}{2}.$$

So the condition is satisfied, and $$df_x(h)=L(h)=(Ax,h).$$

By definition of the gradient, this means $$\nabla f(x)=Ax.$$

-