Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Find the line y = ax+b which provides the best- t (that is, the least-squares fit) for the data:

(x; y) = (-3; 11); (1;-17); (-2; 3); (4;-1)

SOLUTION: The initial (inconsistent) system of equations is

$\begin{pmatrix}\ -3 &1 \\ 1&1 \\ -2&1\\4&1 \end{pmatrix}\begin{pmatrix} a\\b \end{pmatrix} = \begin{pmatrix} 11\\-17\\3\\-1 \end{pmatrix} $

My question is where did all those 1 came from(the column of 1's) if they arent randomly generated numbers how do you find them? Thank you

share|improve this question
add comment

2 Answers

up vote 2 down vote accepted

The column of $1$'s are added to the design matrix because you want to fit a straight line with an intercept. They are not randomly generated. You should add this column every time you want to fit a line of the form $$ y=ax+b. $$ If somehow you believe that there is no intercept in the true model, i.e. you believe the true model is of the form $$ y=ax, $$ then you can delete the column of $1$'s and use just the observations of $x$. If you write $$ Y=\begin{pmatrix} 11\\-17\\3\\-1 \end{pmatrix} \text{ and } X=\begin{pmatrix}\ -3 &1 \\ 1&1 \\ -2&1\\4&1 \end{pmatrix}=(X_1,X_2) $$ the regression coefficients is a vector: $$ \begin{pmatrix} \hat a\\\hat b \end{pmatrix} =(XX')^{-1}X'Y $$ If you omit the column of $1$'s, you only get a scalar, which estimates the model $y=ax$: $$ \tilde a=(X_1X_1')^{-1}X_1'Y. $$

share|improve this answer
add comment

They come from the $b$ in your equation $y=ax+b$. Just compute the first row of your system of equations then you'll see: $-3a+ 1b = 11$

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.