Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It only takes a minute to sign up.
Sign up to join this community
Find the line y = ax+b which provides the best- t (that is, the least-squares fit) for the data:
(x; y) = (-3; 11); (1;-17); (-2; 3); (4;-1)
SOLUTION: The initial (inconsistent) system of equations is
$\begin{pmatrix}\ -3 &1 \\ 1&1 \\ -2&1\\4&1 \end{pmatrix}\begin{pmatrix} a\\b \end{pmatrix} = \begin{pmatrix} 11\\-17\\3\\-1 \end{pmatrix} $
My question is where did all those 1 came from(the column of 1's) if they arent randomly generated numbers how do you find them? Thank you
The column of $1$'s are added to the design matrix because you want to fit a straight line with an intercept. They are not randomly generated. You should add this column every time you want to fit a line of the form $$ y=ax+b. $$ If somehow you believe that there is no intercept in the true model, i.e. you believe the true model is of the form $$ y=ax, $$ then you can delete the column of $1$'s and use just the observations of $x$. If you write $$ Y=\begin{pmatrix} 11\\-17\\3\\-1 \end{pmatrix} \text{ and } X=\begin{pmatrix}\ -3 &1 \\ 1&1 \\ -2&1\\4&1 \end{pmatrix}=(X_1,X_2) $$ the regression coefficients is a vector: $$ \begin{pmatrix} \hat a\\\hat b \end{pmatrix} =(XX')^{-1}X'Y $$ If you omit the column of $1$'s, you only get a scalar, which estimates the model $y=ax$: $$ \tilde a=(X_1X_1')^{-1}X_1'Y. $$
They come from the $b$ in your equation $y=ax+b$. Just compute the first row of your system of equations then you'll see: $-3a+ 1b = 11$
