# Let $r = b - Ax$ be the corresponding residual vector. Which of the following three vectors is a possible value for $r$? Why?

Let $x$ be the solution to the linear least squares problem $Ax \approx b$, where

$A = \begin{bmatrix} 1 & 0 \\ 1 & 1 \\ 1 & 2 \\ 1 & 3 \end{bmatrix}$

Let $r = b - Ax$ be the corresponding residual vector. Which of the following three vectors is a possible value for $r$? Why?

$(a) \begin{bmatrix} 1 \\ 1 \\ 1 \\ 1\end{bmatrix} \: \: \: \: (b) \begin{bmatrix} -1 \\ -1 \\ 1 \\ 1\end{bmatrix} \: \:\: \: (c) \begin{bmatrix} -1 \\ 1 \\ 1 \\ -1\end{bmatrix}$

So thats the question. I do know how to apply least squares using normal equations etc. I just do not have a clue how to think about this one.. Hoping someone could shed some light.

Hint: From the formulation for least squares, $Ax$ is "closest" to $b$ precisely when $Ax$ is the orthogonal projection of $b$ on the range of $A$. Therefore, whatever is left over, the residual $r$, must be orthogonal to the range of $A$.