In this answer Derivation of soft thresholding operator how can I derive that $\nabla(||x-b||_2^2)=b-x$?


The following fact will be helpful in your problem.

$$\frac{\partial}{\partial {\bf x}}||{\bf x}||_2^2 = \frac{\partial}{\partial {\bf x}}||{\bf x}^T{\bf x}||_2 = 2{\bf x}$$

To obtain the above result (namely the last equality), write the definition of the $2$-norm of ${\bf x}^T{\bf x}$ and differentiate wrt to each $x_i$. Now, just replace ${\bf x}$ with ${\bf x}-{\bf a}$ and carefully step through the process again to obtain $\frac{\partial}{\partial {\bf x}}||{\bf x}-{\bf a}||_2^2$.

  • $\begingroup$ but it says that the solution is $a-x$ not $x-a$ $\endgroup$ – volperossa Mar 26 '14 at 23:13
  • $\begingroup$ I don't see anywhere in the link you provided that states that '$\nabla||x-b||_2^2 = b-x$' (which I don't believe to be true). All that is stated is equivalence of optimality conditions. $\endgroup$ – Erik M Mar 26 '14 at 23:31
  • $\begingroup$ here: $0 \in \nabla(\|x-z\|^2_2) + \partial(\lambda\|z\|_1) \Leftrightarrow 0 \in z-x + \lambda\partial\|z\|_1$ $\endgroup$ – volperossa Mar 26 '14 at 23:39
  • $\begingroup$ $(0 \in \nabla(\|x-z\|^2_2) + \partial(\lambda\|z\|_1) \Leftrightarrow 0 \in z-x + \lambda\partial\|z\|_1) \not\Rightarrow \nabla(\|x-z\|^2_2) = z-x$ $\endgroup$ – Erik M Mar 26 '14 at 23:41
  • $\begingroup$ :o so...why z-x?! :o $\endgroup$ – volperossa Mar 26 '14 at 23:46


$\nabla(||x-b||_2^2) = \nabla(||b-x||_2^2) = \nabla[(x-b)^T(x-b)] = \nabla[(x^T-b^T)(x-b)] = \nabla[x^Tx-x^Tb-b^Tx-b^Tb] = 2x-b-b=2(x-b)$

  1. I think there should be $2$ there.
  2. I think $x$ should be positive since whatever you expand it, $x^x$ must be positive.

You can see the following vedio: (a similar derivation)


(roughly around 40:00)


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.