2
$\begingroup$

Use the least square approximation to find the closest line (the line of "Best Fit") to the points:

$$(-6,-1), \quad (-2,2), \quad (1,1), \quad (7,6)$$

I'm attempting to use the least squares approximation formulation that is as follows:

$$A^TAx = A^Tb$$

However, I'm confused because I'm given four vectors. Does that mean I can use the first three vectors $(-6,-1),(-2,2),(1,1)$ to create my matrix A, and then use the last vector $(7,6)$ for the "$b$" value? I understand the process is very straight forward once proper substitution has been done, but I tried the method I described above and got the wrong answer.Thank you guys.

$\endgroup$
2
$\begingroup$

You are looking for an equation $y=mx+c$. Ideally it would pass through all of the given points: that is, you would have $-6m+c = -1$, and similarly for other points. This is a set of four equations with two unknowns $m,c$. Its matrix representation is $$ \begin{pmatrix} -6 & 1\\ -2 & 1 \\ 1 & 1 \\ 7 & 1 \end{pmatrix} \begin{pmatrix} m \\ c \end{pmatrix} = \begin{pmatrix} -1 \\ 2 \\ 1 \\6 \end{pmatrix} $$ These are $A$ and $b$ you are looking for. Typically, the system $Ax=b$ has no solutions (there is no line through all the points); this is why we solve $A^TAx=A^Tb$ instead, obtaining $x$ that minimizes the residual $\|Ax-b\|^2$.

$\endgroup$
  • 1
    $\begingroup$ thank you so much for that. That was a very helpful answer. $\endgroup$ – Arthur Smith May 12 '15 at 5:46
1
$\begingroup$

The trial function is $$ y(x) = c_{0} + c_{1} x. $$ As noted by @user147263,you have the linear system $$ \begin{align} \mathbf{A} c & = y\\ % \left[ \begin{array}{rr} 1 & -6 \\ 1 & -2 \\ 1 & 1 \\ 1 & 7 \\ \end{array} \right] % \left[ \begin{array}{r} c_{0} \\ c_{1} \\ \end{array} \right] % &= % \left[ \begin{array}{r} -1 \\ 2 \\ 1 \\ 6 \end{array} \right]. % \end{align} $$

Your choice for solution is the normal equations $$ \begin{align} % \mathbf{A}^{*} \mathbf{A} c &= \mathbf{A}^{*}y \\ % \left[ \begin{array}{cc} 4 & 0 \\ 0 & 90 \end{array} \right] % % \left[ \begin{array}{r} c_{0} \\ c_{1} \\ \end{array} \right] % &= % \left[ \begin{array}{r} 8 \\ 45 \end{array} \right] . % \end{align} $$ The solution is $$ \begin{align} c &= \left( \mathbf{A}^{*} \mathbf{A} \right)^{-1} \mathbf{A}^{*} y \\ % \left[ \begin{array}{r} c_{0} \\ c_{1} \\ \end{array} \right] % &= % \left[ \begin{array}{cc} \frac{1}{4} & 0 \\ 0 & \frac{1}{90} \\ \end{array} \right] % \left[ \begin{array}{r} 8 \\ 45 \end{array} \right] \\ % &= % \left[ \begin{array}{c} 2 \\ \frac{1}{2} \end{array} \right] . % \end{align} $$ The solution function is $$ y(x) = 2 + \frac{1}{2} x. $$ The residual error vector is $$ r = \mathbf{A}c - 7 = \left[ \begin{array}{r} 0 \\ -1 \\ \frac{3}{2} \\ -\frac{1}{2} \end{array} \right] $$ with a total error of $r^{2} = \frac{7}{2}.$

The solution is plotted against the data points below.

Data and solution

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.