# gradient norm of a simple function

In this answer Derivation of soft thresholding operator how can I derive that $\nabla(||x-b||_2^2)=b-x$?

$$\frac{\partial}{\partial {\bf x}}||{\bf x}||_2^2 = \frac{\partial}{\partial {\bf x}}||{\bf x}^T{\bf x}||_2 = 2{\bf x}$$

To obtain the above result (namely the last equality), write the definition of the $2$-norm of ${\bf x}^T{\bf x}$ and differentiate wrt to each $x_i$. Now, just replace ${\bf x}$ with ${\bf x}-{\bf a}$ and carefully step through the process again to obtain $\frac{\partial}{\partial {\bf x}}||{\bf x}-{\bf a}||_2^2$.

• but it says that the solution is $a-x$ not $x-a$ – volperossa Mar 26 '14 at 23:13
• I don't see anywhere in the link you provided that states that '$\nabla||x-b||_2^2 = b-x$' (which I don't believe to be true). All that is stated is equivalence of optimality conditions. – Erik M Mar 26 '14 at 23:31
• here: $0 \in \nabla(\|x-z\|^2_2) + \partial(\lambda\|z\|_1) \Leftrightarrow 0 \in z-x + \lambda\partial\|z\|_1$ – volperossa Mar 26 '14 at 23:39
• $(0 \in \nabla(\|x-z\|^2_2) + \partial(\lambda\|z\|_1) \Leftrightarrow 0 \in z-x + \lambda\partial\|z\|_1) \not\Rightarrow \nabla(\|x-z\|^2_2) = z-x$ – Erik M Mar 26 '14 at 23:41
• :o so...why z-x?! :o – volperossa Mar 26 '14 at 23:46

$\nabla(||x-b||_2^2)=b-x$

Proof:
$\nabla(||x-b||_2^2) = \nabla(||b-x||_2^2) = \nabla[(x-b)^T(x-b)] = \nabla[(x^T-b^T)(x-b)] = \nabla[x^Tx-x^Tb-b^Tx-b^Tb] = 2x-b-b=2(x-b)$

1. I think there should be $2$ there.
2. I think $x$ should be positive since whatever you expand it, $x^x$ must be positive.

You can see the following vedio: (a similar derivation)

(roughly around 40:00)