Least square, Signifiance of plane we are projecting on I have points  $(1,1) (2,2)$ and  $(3,2)$. Now I wish use least-squares to fit a line through it.
The equations I get,
$$
C+D=1$$ $$
C+2D=2$$ $$
C+3D=2$$
Now, of course there exists no solution to this system($Ax=b$).
We are obtaining the best fit hyperplanein $\mathbb R^{2}$(,i.e., a line) for the points. This plane has the closest projections of vector $b$.
Questions:


*

*I can't understand what the plane in $\mathbb R^{3}$ represents on which we are projecting on. What's does it signify?

*How the the plane in $\mathbb R^{3}$ relate to the best-fit line we obtain?

 A: The linear system is
$$
\begin{align}
%
  \mathbf{A} c &= b \\[2pt]
%
\left[ \begin{array}{cc}
  1 & 1 \\ 1 & 2 \\ 1 & 3
\end{array} \right]
%
\left[ \begin{array}{c}
  C \\ D
\end{array} \right]
%
  & =
%
\left[ \begin{array}{c}
  1 \\ 2 \\ 2
\end{array} \right]
%
\end{align}
$$
The solution is 
$$
\left[ \begin{array}{c}
  C \\ D
\end{array} \right]_{LS}
=
\frac{1}{6}
\left[ \begin{array}{c}
  4 \\ 3
\end{array} \right]
$$
The matrix $\mathbf{A}$ is a map which takes vectors from the domain $\mathbb{C}^{n}$ and maps them to vectors in the image $\mathbb{C}^{m}$.
The matrix $\mathbf{A}$ is a map which connects vectors from the solution space $\mathbb{C}^{n}$ and to vectors in the data space $\mathbb{C}^{m}$.
If you are doing a linear regression with 10 data points, the solution space is $\mathbb{C}^{2}$ where your slope and intercept live. Your data space is $\mathbb{C}^{10}$ where the measurements live.
More about the fundamental projectors: Least squares solutions and the orthogonal projector onto the column space
