# Least squares projection

Let $$x_1$$ and $$x_2$$ such that $$x_1\ne x_2$$ are two least-squares solutions to the equation $$Ax=b$$.

Prove that $$Ax_1=Ax_2$$, meaning that the projection of $$b$$ on $$Col(A)$$ is unique.

My way: I know that the norm of the residual vectors is the same. I'd like to show that $$Ax_1-b$$=$$Ax_2-b$$ but I stuck here.

• Just to make sure I'm understanding the question properly: $x_1$ and $x_2$ are both least-squares solutions to the equation $Ax=b$, correct? – Noble Mushtak Dec 9 '19 at 17:38
• @NobleMushtak Yes – Ro168 Dec 9 '19 at 17:41

First, we know that $$Ax_1-b$$ and $$Ax_2-b$$ are in the left nullspace of $$A$$ (i.e. the orthogonal complement of the image of $$A$$). This is because, if $$Ax_1-b$$ was not in the left nullspace of $$A$$, then the residual $$\|Ax_1-b\|$$ would not be minimized and $$x_1$$ would not be a least-squares solution.
Now, notice that: $$b=Ax_1-(Ax_1-b)=Ax_2-(Ax_2-b)\implies Ax_1-Ax_2=(Ax_1-b)-(Ax_2-b)$$ The left side of this equation is clearly is in the image of $$A$$, since $$Ax_1$$ and $$Ax_2$$ are both in the image of $$A$$. Meanwhile, the right side of this equation is in the left nullspace of $$A$$, since $$Ax_1-b$$ and $$Ax_2-b$$ are both in the left nullspace of $$A$$. Thus, this quantity is in both the image and the left nullspace of $$A$$, but since these two subspaces are orthogonal complements of each other, the only way a vector can be in both of them at the same time is if that vector is $$\vec 0$$.
Therefore: $$Ax_1-Ax_2=\vec 0\implies Ax_1=Ax_2$$