In this answer Derivation of soft thresholding operator how can I derive that $\nabla(||x-b||_2^2)=b-x$?
$\begingroup$
$\endgroup$
$\begingroup$
$\endgroup$
The following fact will be helpful in your problem.
$$\frac{\partial}{\partial {\bf x}}||{\bf x}||_2^2 = \frac{\partial}{\partial {\bf x}}||{\bf x}^T{\bf x}||_2 = 2{\bf x}$$
To obtain the above result (namely the last equality), write the definition of the $2$-norm of ${\bf x}^T{\bf x}$ and differentiate wrt to each $x_i$. Now, just replace ${\bf x}$ with ${\bf x}-{\bf a}$ and carefully step through the process again to obtain $\frac{\partial}{\partial {\bf x}}||{\bf x}-{\bf a}||_2^2$.
-
$\begingroup$ but it says that the solution is $a-x$ not $x-a$ $\endgroup$ – volperossa Mar 26 '14 at 23:13
-
$\begingroup$ I don't see anywhere in the link you provided that states that '$\nabla||x-b||_2^2 = b-x$' (which I don't believe to be true). All that is stated is equivalence of optimality conditions. $\endgroup$ – Erik M Mar 26 '14 at 23:31
-
$\begingroup$ here: $0 \in \nabla(\|x-z\|^2_2) + \partial(\lambda\|z\|_1) \Leftrightarrow 0 \in z-x + \lambda\partial\|z\|_1$ $\endgroup$ – volperossa Mar 26 '14 at 23:39
-
$\begingroup$ $(0 \in \nabla(\|x-z\|^2_2) + \partial(\lambda\|z\|_1) \Leftrightarrow 0 \in z-x + \lambda\partial\|z\|_1) \not\Rightarrow \nabla(\|x-z\|^2_2) = z-x$ $\endgroup$ – Erik M Mar 26 '14 at 23:41
-
$\begingroup$
$\endgroup$
$\nabla(||x-b||_2^2)=b-x$
Proof:
$\nabla(||x-b||_2^2) = \nabla(||b-x||_2^2) = \nabla[(x-b)^T(x-b)] =
\nabla[(x^T-b^T)(x-b)] = \nabla[x^Tx-x^Tb-b^Tx-b^Tb] = 2x-b-b=2(x-b)$
- I think there should be $2$ there.
- I think $x$ should be positive since whatever you expand it, $x^x$ must be positive.
You can see the following vedio: (a similar derivation)
https://www.youtube.com/watch?v=jbdyR7ESOh4&list=PL7y-1rk2cCsDOv91McLOnV4kExFfTB7dU&index=6
(roughly around 40:00)
answered May 29 '15 at 22:16
