how to project optimal parameters on to feasible region Hi: I'm trying to understand the concept of projection and I created a toy example that might help me to do that. 
Suppose that I have a non-linear optimization with 3 parameters theta_1, theta_2 and theta_3. The actual problem requires that theta_1, theta_2 and theta_3 be between 0 and 1 and that their sum is equal 1.0.
So, now suppose I optimize without any of the constraints and get back the optimal unconstrained parameters theta_1 = 1.992, theta_2 = 0.575 and theta_3 = 0.487.
So, now suppose that I want to project those optimal parameters into the feasible region so the projected parameters are as close to the original ones
as possible.
I think understand the concept of projections when I read about it in
textbooks, both in the vector case and matrix case.
1) vector case: To project a vector $b$  on to the subspace spanned by another vector $a$, it can be shown that the point through the origin
and along the point a,  where the distance between from b to a is minimized is $\frac{a^{T}b}{a^{T}a} \times a$. The derivation made sense to me but
I won't show it here. ( It's pretty standard result obviously ).
2) matrix case: if $X$ is $n \times k$ and $\beta$ is $k \times 1$, and we have $y = X \beta$ where $\beta$ is unknown, then $\hat{\beta} = 
 (X^{\prime}X)^{-1}X^{\prime}y$ and $\hat{y} =   X \hat{\beta}  = X (X^{\prime}X)^{-1}X^{\prime}y$ is the projection of y onto the spaced spanned by the column space of $X$. The derivation of that made sense
and the derivation is pretty much the same as 1) where $b = y$ and 
$a = X\beta$.
But I don't know how to apply the results of 1) or 2) to the toy optimization  problem that I described. Someone told me that the answer is that the projected coefficients are the original coefficients divided by their sum but I don't see how to obtain that.
I think if I can understand how one solves the toy problem, then I'll understand the application of projections rather than just the formulae.
Thank you very much for any explanations using either 1 or 2.
Also, if anyone could tell me if the coefficients divided by their
sum has any meaning in terms of a projection, that would be appreciated 
also.
 A: Your problem is essentially: given a point $x \in \mathbb{R}^n$ and a convex set $C \subset \mathbb{R}^n$, how do I project $x$ onto $C$?  There's lots of work on this; you want a projection operator.  The definition of the projection of $x$ onto $C$ is the point $y \in C$ that minimizes the Euclidean distance $\|x-y\|$.
In your case, the convex set $C$ has a special form: it is a box.  In particular, in your example, it is a unit box: $C=[0,1]^n$.  For this special case, projection is especially easy.  In particular, the projection of $x$ onto $C$ is the vector $y$ defined as
$$y_i = \max(0, \min(x_i, 1)).$$
In other words, you clamp each coordinate to be within the range $[0,1]$: if it is already in $[0,1]$, you do nothing; if it is larger than $1$, you clamp it to $1$; if it is smaller than $0$, you clamp it to $0$.
The same works with any box: you clamp each entry, separately, to the interval it is supposed to be within.
Once you know about this, look up projected gradient descent.
