Prove that $\nabla\langle Ax,Ax\rangle = 2A^TAx$ Prove that $\nabla\langle Ax,Ax\rangle = 2A^TAx$.
My book uses this property to prove the $2-norm$ of a matrix $A$ is the square root of the spectral radius of $A^TA$. That is 
$$||A||_2 = \sqrt{r(A^TA)}$$
where $r(A)$ is the spectral radius. How would one go about proving this?
This isn't a homework question. I would just like to know for completeness of the proof. T
Any help and comments is appreciated. Thank you in advance. 
 A: Expand it in each component,
$$\begin{align}\left(\nabla\langle Ax,Ax\rangle\right)_i&=\partial_i\left(\sum_j\left(\sum_kA_{jk}x_k\right)\left(\sum_lA_{jl}x_l\right)\right)\\
&=\sum_j\left(A_{ji}\left(\sum_lA_{jl}x_l\right)+\left(\sum_kA_{jk}x_k\right)A_{ji}\right)\\
&=2\sum_jA_{ji}\left(\sum_kA_{jk}x_k\right)\end{align}$$
Or,
$$\nabla\langle Ax,Ax\rangle=2A^TAx$$
A: Using the indice summation convention for repeated indices and letting $v=\nabla\langle Ax,Ax\rangle$, we have
$$v_i=\partial_i(A_{jk}x_kA_{jl}x_l)=A_{jk}\delta_{ik}A_{jl}x_l+A_{jk}x_kA_{jl}\delta_{il}=
A_{ji}A_{jl}x_l+A_{jk}x_kA_{ji}=2A_{ij}^TA_{jk}x_k=(2A^TAx)_i,$$
so $\nabla\langle Ax,Ax\rangle=2A^TAx$.
A: Let's do it: first let's remember that $$\langle Ax, y \rangle = \sum_i (Ax)_iy_i = \sum_{i,j} A_{ij} x_j y_i  = \sum_{i,j} x_j A^T_{ji} y_i = \langle x, A^T y \rangle$$
Thus $\langle Ax, Ax \rangle = \langle x, A^TA x \rangle$, following the suggestion of pppqqq let's write $A^TA =: B$. Then $\langle x, Bx \rangle = \sum_{i,j} b_{ij}x_ix_j$. Let's compute the gradient of this at a point $x$. Well, let's determine the $\alpha^{th}$ component, by definition this is: $$\partial_{\alpha} \sum_{i,j} b_{ij}x_ix_j = \sum_{j \neq \alpha} b_{\alpha j} x_j + \sum_{i \neq \alpha} b_{i \alpha} x_i + \sum_{i = j = \alpha} 2b_{\alpha \alpha} x_\alpha$$We may split up the last term into $b_{\alpha \alpha} x_\alpha + b_{\alpha \alpha} x_\alpha$, so we have$$= \sum_j b_{\alpha j} x_j + \sum_i b_{i \alpha} x_i = (Bx)_\alpha + \langle B(e_\alpha), x \rangle$$Now, we notice that in our case $B$ is its own transpose, so we have $$ = \sum_j b_{\alpha j}x_j + \sum_i b_{\alpha i} x_i = 2 \sum_j b_{\alpha j} x_j = 2(Bx)_\alpha$$as desired. 
