# Using least squares regression for line of best fit

Use the least square approximation to find the closest line (the line of "Best Fit") to the points:

$$(-6,-1), \quad (-2,2), \quad (1,1), \quad (7,6)$$

I'm attempting to use the least squares approximation formulation that is as follows:

$$A^TAx = A^Tb$$

However, I'm confused because I'm given four vectors. Does that mean I can use the first three vectors $(-6,-1),(-2,2),(1,1)$ to create my matrix A, and then use the last vector $(7,6)$ for the "$b$" value? I understand the process is very straight forward once proper substitution has been done, but I tried the method I described above and got the wrong answer.Thank you guys.

You are looking for an equation $y=mx+c$. Ideally it would pass through all of the given points: that is, you would have $-6m+c = -1$, and similarly for other points. This is a set of four equations with two unknowns $m,c$. Its matrix representation is $$\begin{pmatrix} -6 & 1\\ -2 & 1 \\ 1 & 1 \\ 7 & 1 \end{pmatrix} \begin{pmatrix} m \\ c \end{pmatrix} = \begin{pmatrix} -1 \\ 2 \\ 1 \\6 \end{pmatrix}$$ These are $A$ and $b$ you are looking for. Typically, the system $Ax=b$ has no solutions (there is no line through all the points); this is why we solve $A^TAx=A^Tb$ instead, obtaining $x$ that minimizes the residual $\|Ax-b\|^2$.
The trial function is $$y(x) = c_{0} + c_{1} x.$$ As noted by @user147263,you have the linear system \begin{align} \mathbf{A} c & = y\\ % \left[ \begin{array}{rr} 1 & -6 \\ 1 & -2 \\ 1 & 1 \\ 1 & 7 \\ \end{array} \right] % \left[ \begin{array}{r} c_{0} \\ c_{1} \\ \end{array} \right] % &= % \left[ \begin{array}{r} -1 \\ 2 \\ 1 \\ 6 \end{array} \right]. % \end{align}
Your choice for solution is the normal equations \begin{align} % \mathbf{A}^{*} \mathbf{A} c &= \mathbf{A}^{*}y \\ % \left[ \begin{array}{cc} 4 & 0 \\ 0 & 90 \end{array} \right] % % \left[ \begin{array}{r} c_{0} \\ c_{1} \\ \end{array} \right] % &= % \left[ \begin{array}{r} 8 \\ 45 \end{array} \right] . % \end{align} The solution is \begin{align} c &= \left( \mathbf{A}^{*} \mathbf{A} \right)^{-1} \mathbf{A}^{*} y \\ % \left[ \begin{array}{r} c_{0} \\ c_{1} \\ \end{array} \right] % &= % \left[ \begin{array}{cc} \frac{1}{4} & 0 \\ 0 & \frac{1}{90} \\ \end{array} \right] % \left[ \begin{array}{r} 8 \\ 45 \end{array} \right] \\ % &= % \left[ \begin{array}{c} 2 \\ \frac{1}{2} \end{array} \right] . % \end{align} The solution function is $$y(x) = 2 + \frac{1}{2} x.$$ The residual error vector is $$r = \mathbf{A}c - 7 = \left[ \begin{array}{r} 0 \\ -1 \\ \frac{3}{2} \\ -\frac{1}{2} \end{array} \right]$$ with a total error of $r^{2} = \frac{7}{2}.$ 