# Least Squares with Unit Simplex Constraint Variation

I have to solve the following least square problem:

$$\hat{x} = \arg \min_{x \in S} \|Ax - b\|^2$$

If $S = \mathbb{R}^n$, then the solution is given by

$$\hat{x} = (A^TA)^{-1}A^Tb$$

supposing that $(A^TA)^{-1}$ exists.

What if $$S = \left\{x \in \mathbb{R}^n : \sum_{i=1}^n x_i = 1, x_i \in [0, 1] ~\forall i \in \{1, \ldots, n\}\right\}?$$

** Addition **

$\|\cdot\|$ is the euclidean norm (2-norm)

$A \in \mathbb{R}^{n\times n}$ and $b \in \mathbb{R}^n$

## 1 Answer

The trick is to observe that in the context of the problem the following 2 problems are equivalent:

\begin{alignat*}{3} \arg \min_{x} & \quad & \frac{1}{2} \left\| A x - b \right\|_{2}^{s} \\ \text{subject to} & \quad & \boldsymbol{1}^{T} x = 1 \\ & \quad & {x}_{i} \in \left[ 0, 1 \right], \; \forall i \end{alignat*}

and

\begin{alignat*}{3} \arg \min_{x} & \quad & \frac{1}{2} \left\| A x - b \right\|_{2}^{s} \\ \text{subject to} & \quad & \boldsymbol{1}^{T} x = 1 \\ & \quad & x \succeq 0 \end{alignat*}

The second one is basically Least Squares constrained to the Unit Simplex.
There is no closed form solution to that but it can be solved using Projected Gradient Descent.
For a full solution you can find in my answer to Least Squares with Unit Simplex Constraint.