1
$\begingroup$

In this answer Derivation of soft thresholding operator how can I derive that $\nabla(||x-b||_2^2)=b-x$?

$\endgroup$

2 Answers 2

2
$\begingroup$

The following fact will be helpful in your problem.

$$\frac{\partial}{\partial {\bf x}}||{\bf x}||_2^2 = \frac{\partial}{\partial {\bf x}}||{\bf x}^T{\bf x}||_2 = 2{\bf x}$$

To obtain the above result (namely the last equality), write the definition of the $2$-norm of ${\bf x}^T{\bf x}$ and differentiate wrt to each $x_i$. Now, just replace ${\bf x}$ with ${\bf x}-{\bf a}$ and carefully step through the process again to obtain $\frac{\partial}{\partial {\bf x}}||{\bf x}-{\bf a}||_2^2$.

$\endgroup$
7
  • $\begingroup$ but it says that the solution is $a-x$ not $x-a$ $\endgroup$
    – volperossa
    Mar 26, 2014 at 23:13
  • $\begingroup$ I don't see anywhere in the link you provided that states that '$\nabla||x-b||_2^2 = b-x$' (which I don't believe to be true). All that is stated is equivalence of optimality conditions. $\endgroup$
    – Erik M
    Mar 26, 2014 at 23:31
  • $\begingroup$ here: $0 \in \nabla(\|x-z\|^2_2) + \partial(\lambda\|z\|_1) \Leftrightarrow 0 \in z-x + \lambda\partial\|z\|_1$ $\endgroup$
    – volperossa
    Mar 26, 2014 at 23:39
  • $\begingroup$ $(0 \in \nabla(\|x-z\|^2_2) + \partial(\lambda\|z\|_1) \Leftrightarrow 0 \in z-x + \lambda\partial\|z\|_1) \not\Rightarrow \nabla(\|x-z\|^2_2) = z-x$ $\endgroup$
    – Erik M
    Mar 26, 2014 at 23:41
  • $\begingroup$ :o so...why z-x?! :o $\endgroup$
    – volperossa
    Mar 26, 2014 at 23:46
2
$\begingroup$

$\nabla(||x-b||_2^2)=b-x$

Proof:
$\nabla(||x-b||_2^2) = \nabla(||b-x||_2^2) = \nabla[(x-b)^T(x-b)] = \nabla[(x^T-b^T)(x-b)] = \nabla[x^Tx-x^Tb-b^Tx-b^Tb] = 2x-b-b=2(x-b)$

  1. I think there should be $2$ there.
  2. I think $x$ should be positive since whatever you expand it, $x^x$ must be positive.

You can see the following vedio: (a similar derivation)

https://www.youtube.com/watch?v=jbdyR7ESOh4&list=PL7y-1rk2cCsDOv91McLOnV4kExFfTB7dU&index=6

(roughly around 40:00)

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .