# Least squares solutions and orthogonal projection?

I found the least squares solution for the following inconsistent system of equations:

$x_1 - x_2 = 0$

$x_1 + x_2 = 5$

$-x_1 + x_2 = 2$ , which turned out to be $\begin{bmatrix} 2\\ 3\\ \end{bmatrix}$.

The next part of this question asks me to use this result to find the orthogonal projection of the vector $\begin{bmatrix} 0\\ 5\\ 2\\ \end{bmatrix}$ on the span of the vectors $\begin{bmatrix} 1\\ 1\\ -1\\ \end{bmatrix}$ and $\begin{bmatrix} -1\\ 1\\ 1\\ \end{bmatrix}$.

To do this, is it simply the following calculation?

$\begin{bmatrix} 1&-1\\ 1&1\\ -1&1\\ \end{bmatrix}$ $\begin{bmatrix} 3\\ 2\\ \end{bmatrix}$? When I do this, I get $\begin{bmatrix} 1\\ 5\\ -1\\ \end{bmatrix}$. Is this the correct answer?

• I presume you meant $(2,3)^T$ in the last line? – copper.hat Jul 24 '16 at 4:13

## 1 Answer

Note that ${\cal R} A = \operatorname{sp} \{ (1,1,-1)^T , (-1,1,1)^T \}$.

The least squares solution $\hat{x}$ minimises $\|Ax-b\|^2$.

In particular, the point $A \hat{x}$ is the closest point to $b$ in ${\cal R} A$, so we have $(A \hat{x} -b) \bot {\cal R}A$, that is, $A \hat{x}$ is the orthogonal projection of $b$ onto ${\cal R} A$.

Computing gives $A {\hat{x}} = (-1,5,1)^T$.

It is straightforward to check that $A^T (A \hat{x} -b) = 0$.