Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

How is it possible that the matrix equation $\textbf{Ax} = \textbf{b}$ has a least squares solution that is a line through the origin? Isn't it the point closest to the vector $\textbf{x}$ in the column space of $\textbf{A}$ thus always a one-dimensional space?

share|cite|improve this question
Who says it's a line through the origin? – Will Jagy Nov 16 '12 at 21:07
up vote 2 down vote accepted

The least squares solution $\hat{x}$ to the system $Ax = b$ can be found by solving the normal equations $A^T A \hat{x} = A^T b$. It is known that $\mathrm{rank}(A^TA) = \mathrm{rank}(A)$, so by rank-nullity, the set of solutions to this system forms an affine space (a hyperplane) which does not necessarily pass through the origin, but which, if translated to the origin, forms a subspace of dimension $n - \mathrm{rank}(A)$, where $n$ is the number of columns of $A$.

Given this, there are several things you've mentioned I need to address. Firstly, the least squares solution does not need to be a subspace. Furthermore, even if it were a subspace, it does not need to be one-dimensional (a line), as the number of least squares solutions depends on the size of the null space of $A$, i.e. there can be many choices of $\hat{x}$ such that $A\hat{x}$ is the closest point to $b$ in the column space of $A$.

share|cite|improve this answer
Hmm, it looks like I misinterpreted that the least squares solution was in the column space but indeed, it isn't at all. – Joj Nov 17 '12 at 9:22

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.