Lasso ADMM with Positive Constraint I am solving the Lasso with ADMM. 
The problem is given by:
$$ \begin{align*}
\arg \min_{x} & \left\{ {\left\| A x - b \right\|}_{2}^{2} + \lambda {\left\| x \right\|}_{1} \right\} \\ 
\text{ s.t. } & x \succeq 0
\end{align*} $$
I am trying to tackle the inequality constraint by another equality constraint.
$$x = \epsilon^2$$
The questions are


*

*Does ADMM support two equality constraint? 

*Is there another approach to solve Lasso with positive constraint?

 A: This is pretty simple variation of the LASSO as the constraint is easy to solve (Namely its projection has closed form solution which is simple).
Let's define $ f \left( x \right) = \frac{1}{2} \left\| A x - b \right\|_{2}^{2} + \lambda \left\| x \right\|_{1} $.
I solved it using 3 methods.
Projected Sub Gradient Method
The Sub Gradient is given by:
$$ \partial f \left( x \right) = {A}^{T} \left( A x - b \right) + \lambda \operatorname{sign} \left( x \right) $$
The projection onto $ \mathbb{R}_{+} $ is given by:
$$ \operatorname{Proj}_{ \mathbb{R}_{+} } \left( x \right) = \max \left\{ x, 0 \right\} $$
The iteration is simple:
$$ {x}^{k + 1} = \operatorname{Proj}_{ \mathbb{R}_{+} } \left( {x}^{k} - \alpha \partial f \left( x \right) \right) $$
Where $ \alpha $ is the step size of the algorithm.
Proximal Gradient Method
The Proximal Gradient Method for LASSO is well known.
One could see it as a generalization of the Gradient Method.
Hence by adding the Projection Step as above into its iteration solves the problem (Faster) as well.
ADMM
There is a large degree of freedom to solve this using the ADMM Framework.
The approach I did was just a small alteration to the classic solution for the LASSO using ADMM. I just applied the projection on the variable which is being threshold.
It doesn't work perfectly (The Objective Function isn't decreasing monotonically) yet it works really nicely and the idea was form the background of ADMM (Dual Decompositions).
This is the result I got validated against CVX:

Remarks


*

*One could use Accelerated Method to farther speed up the Proximal Gradient Method.

*One could formulate the problem using Indicator Function over the non negative orthant. This will yield Classic ADMM for LASSO where the $ x $ iteration will require solving Non Negative Linear Least Squares instead of Linear Least Squares.

*There are some other formulations for Constrained LASSO (Though the one above is specific to the above while those are more general) such as:


*

*Algorithms for Fitting the Constrained LASSO (See SparseReg MATLAB Toolbox).

*Distributed Convex Optimization with Many Convex Constraints.

*Generalized Alternating Direction Method of Multipliers: New Theoretical Insights and Applications.



The code is available at my StackExchange Mathematics 2706108 GitHub Repository.
