# Linear Algebra . Least Squares method

My Problem : Use the least squares method to fit a line into the following data : ( points.. )
xy ; (1,4) (-2,5) (3,-1) (4,1)

Attempt at Solution : I write the equation for a line ; y = kx + b ( where k is the slope and b the intercept )

to find k I use the equation :k = ( n(∑xy) - (∑x)*(∑y) ) /( n(∑x^2)-(∑x)^2 )

and for b :( (∑y) - k(∑x) ) / n

n is the number of pairs ( xy .. )

The simple arithmetic is done, and my answer for the line is y = (-37/26 )x + (57/13)

Do you think this is a viable method? I'm worried about my lack of theory, and whether or not my answer is correct.

It seems to me that this is rather a question of "what method your instructor will accept" rather than "how to do the problem". So you should ask him/her.

However, if I can assume from the tag that you are doing a linear algebra course, and if the least squares method has been suggested, then perhaps the following is what is required.

First try to make the line $y=kx+b$ go through all the given points: we need $$k+b=4\,,\ -2k+b=5\,,\ 3k+b=-1\,,\ 4k+b=1\,.$$ Writing this in matrix form, $$\pmatrix{1&1\cr-2&1\cr3&1\cr4&1\cr}\pmatrix{k\cr b\cr} =\pmatrix{4\cr5\cr-1\cr1\cr}\ .$$ However the system has no solution (check this for yourself) and so there is no line going through the four points. To find the line which fits the points as nearly as possible, in the least squares sense, we write the above as $$A\pmatrix{k\cr b\cr}={\bf y}$$ and solve the normal equations $$A^TA\pmatrix{k\cr b\cr}=A^T{\bf y}\ .$$ It can be shown that if $A$ has linearly independent columns, then the normal equations will always have a unique solution. In this case we have $$\pmatrix{30&6\cr6&4\cr}\pmatrix{k\cr b\cr}=\pmatrix{-5\cr9\cr}\ ,$$ and you can solve this to find the "best" values of $k$ and $b$. They should be the same as those found by the method in your question, but I think you may have an error in your arithmetic.

Comment. This method is superior to the one you have suggested in your post because it is much more flexible. For example you can use exactly the same ideas to find the quadratic $y=a+bx+cx^2$ which best fits the data points, or the curve $y=a\cos x+b\sin x$ which best fits them, and so on.

If, for the time being, you are just concerned by fitting a straight line $y=kx+b$ and you still do not know the matrix formulation of the problem, you can use the formula you wrote in your post.

However, your arithmetic is wrong since $$n=4,\sum _{i=1}^n x_i=6, \sum _{i=1}^n y_i=9, \sum _{i=1}^n x_i y_i=-5, \sum _{i=1}^n x_i^2=30$$ from which, applying your formulas, one obtains $k=-\frac {37}{42}$ and $b=\frac {25}{7}$.

If I may suggest, put on the same plot : your data points, your line, my line and see which one is the closest to your points.