3
$\begingroup$

Define the weighted $ {L}_{2} $ norm $ {\left\| x \right\|}_{2,w} = \sqrt{ \sum_{i = 1}^{n} {w}_{i} {x}_{i}^{2} }$. Find the formula for $ \operatorname{prox}_{\lambda {\left\| \cdot \right\|}_{2,w} }(y)$, where $ \lambda > 0 $.

By definition we have: $$ \operatorname{prox}_{\lambda {\left\| \cdot \right\|}_{2,w}} \left( y \right) = \arg \min_{x} \left( \lambda \sqrt{ \sum_{i = 1}^{n} {w}_{i} {x}_{i}^{2} } + \frac{1}{2} {\left\| x - y \right\|}_{2}^{2} \right) $$

But I have no idea how to proceed from here. Any idea?

$\endgroup$
3
$\begingroup$

It's the usual thing in such situations: set the gradient with respect to $x$ to zero and solve for $x$.

In this case, the $i$-th coordinate of the gradient is $$ \frac{w_ix_i}{\sqrt{W}} + \frac{x_i-y_i}{\lambda}$$ where $W=\sum_{i=1}^nw_ix_i^2$ is the weighted squared 2-norm. Setting the above to zero, we get $$ x_i = \frac{y_i}{\lambda w_i/\sqrt{W}+1}.$$ So these expressions for $i=1,\ldots,n$ define the minimizer $x$.

$\endgroup$
  • $\begingroup$ Maybe to clarify the first term can be obtained by applying the chain rule on $f(g(x)), \cases{f(x) = \sqrt{x}\\ g(x) = \sum_{i=1}^{n}w_i{x_i}^2}$. $\endgroup$ – mathreadler May 3 '17 at 10:15
  • $\begingroup$ I think this is not the closed form answer as $ W $ depends on $ {x}_{i} $. For example if we set $ {w}_{i} = 1 $ for all $ i $ we should get the result of the Proximal Operator of the vanilla $ {L}_{2} $ Norm as in math.stackexchange.com/questions/1681658. But the results are not the same. $\endgroup$ – Royi Aug 27 at 14:25
  • $\begingroup$ Could you address my comment? Thank You. $\endgroup$ – Royi Nov 29 at 10:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.