# Least squares regression problem

I have the matrix $A = \begin{bmatrix} 1 & 2 \\ 2 & 4 \end{bmatrix}$

$\begin{bmatrix} 1 & 2 \\ 2 & 4 \end{bmatrix}x = \begin{bmatrix} 1 \\ 6 \end{bmatrix}$, has no solutions

So I want to use least squares regression to find the best approximation.

So I use the normal equation - $A^tAx = A^tb$

$\begin{bmatrix} 1 & 2 \\ 2 & 4 \end{bmatrix}\begin{bmatrix} 1 & 2 \\ 2 & 4 \end{bmatrix}x = \begin{bmatrix} 1 & 2 \\ 2 & 4 \end{bmatrix}\begin{bmatrix} 1 \\ 6 \end{bmatrix}$

$\begin{bmatrix} 5 & 10 \\ 10 & 20 \end{bmatrix}x = \begin{bmatrix} 13 \\ 26 \end{bmatrix}$

However this if I reduce the augmented matrix

$\begin{bmatrix} 5 & 10 & | & 13 \\ 10 & 20 & | & 26 \end{bmatrix}$

I end up with

$\begin{bmatrix} 1 & 2 & | & 2.6 \\ 0 & 0 & | & 0 \end{bmatrix}$

Which is obviuosly not the solution. So where am I going wrong?

-

Actually you're not wrong. Let $x=(x_1,x_2)$. Think about it geometrically. Your initial system represents two lines : $$x_1+2x_2=1 \ (1)$$ and $$2x_1+4x_2=6 \ (2)$$ The least squares solution would be the point that has the minimum distance from both lines, but since these two lines are parallel, any point in the middle would have the minimum distance. All these points form another parallel line with the equation: $$x_1+2x_2=2.6$$ which is the solution of your least squares problem. If the two lines were not parallel, then you would have an exact solution, since they cross each other in exactly one point.