Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I have the matrix $A = \begin{bmatrix} 1 & 2 \\ 2 & 4 \end{bmatrix}$

$\begin{bmatrix} 1 & 2 \\ 2 & 4 \end{bmatrix}x = \begin{bmatrix} 1 \\ 6 \end{bmatrix}$, has no solutions

So I want to use least squares regression to find the best approximation.

So I use the normal equation - $A^tAx = A^tb$

$\begin{bmatrix} 1 & 2 \\ 2 & 4 \end{bmatrix}\begin{bmatrix} 1 & 2 \\ 2 & 4 \end{bmatrix}x = \begin{bmatrix} 1 & 2 \\ 2 & 4 \end{bmatrix}\begin{bmatrix} 1 \\ 6 \end{bmatrix}$

$\begin{bmatrix} 5 & 10 \\ 10 & 20 \end{bmatrix}x = \begin{bmatrix} 13 \\ 26 \end{bmatrix}$

However this if I reduce the augmented matrix

$\begin{bmatrix} 5 & 10 & | & 13 \\ 10 & 20 & | & 26 \end{bmatrix}$

I end up with

$\begin{bmatrix} 1 & 2 & | & 2.6 \\ 0 & 0 & | & 0 \end{bmatrix}$

Which is obviuosly not the solution. So where am I going wrong?

share|improve this question

1 Answer 1

up vote 3 down vote accepted

Actually you're not wrong. Let $x=(x_1,x_2)$. Think about it geometrically. Your initial system represents two lines : $$x_1+2x_2=1 \ (1)$$ and $$2x_1+4x_2=6 \ (2)$$ The least squares solution would be the point that has the minimum distance from both lines, but since these two lines are parallel, any point in the middle would have the minimum distance. All these points form another parallel line with the equation: $$x_1+2x_2=2.6 $$ which is the solution of your least squares problem. If the two lines were not parallel, then you would have an exact solution, since they cross each other in exactly one point.

share|improve this answer
1  
Nice answer. +1 –  user2468 Apr 19 '12 at 20:23

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.