# Dimension of least squares solution

How is it possible that the matrix equation $\textbf{Ax} = \textbf{b}$ has a least squares solution that is a line through the origin? Isn't it the point closest to the vector $\textbf{x}$ in the column space of $\textbf{A}$ thus always a one-dimensional space?

-
Who says it's a line through the origin? –  Will Jagy Nov 16 '12 at 21:07

The least squares solution $\hat{x}$ to the system $Ax = b$ can be found by solving the normal equations $A^T A \hat{x} = A^T b$. It is known that $\mathrm{rank}(A^TA) = \mathrm{rank}(A)$, so by rank-nullity, the set of solutions to this system forms an affine space (a hyperplane) which does not necessarily pass through the origin, but which, if translated to the origin, forms a subspace of dimension $n - \mathrm{rank}(A)$, where $n$ is the number of columns of $A$.

Given this, there are several things you've mentioned I need to address. Firstly, the least squares solution does not need to be a subspace. Furthermore, even if it were a subspace, it does not need to be one-dimensional (a line), as the number of least squares solutions depends on the size of the null space of $A$, i.e. there can be many choices of $\hat{x}$ such that $A\hat{x}$ is the closest point to $b$ in the column space of $A$.

-
Hmm, it looks like I misinterpreted that the least squares solution was in the column space but indeed, it isn't at all. –  Joj Nov 17 '12 at 9:22