Under the assumptions of the classical simple linear regression model, show that the least squares estimator of the slope is an unbiased estimator of the `true' slope in the model.
Anyone have any ideas for the following questions?
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It only takes a minute to sign up.
Sign up to join this communityUnder the assumptions of the classical simple linear regression model, show that the least squares estimator of the slope is an unbiased estimator of the `true' slope in the model.
Anyone have any ideas for the following questions?
The proof is mostly a matter of a little algebra.
The OLS (ordinary least squares) estimator for $\beta_1$ in the model $$y=\beta_0+\beta_1x+u$$ can be shown to have the form $$\hat{\beta_1}=\frac{\sum(x_i-\bar{x})y_i }{\sum x_i^2-n\bar{x}^2 }$$ Since you didn't say what you've tried, I don't know if you understand how to derive this expression from whatever your book defines $\hat{\beta_1}$ to be.
In any case, taking the conditional expectation with respect to the $x_i$, we get $$E\left[\hat{\beta_1}\left|\right.x_i\right]=\frac{1}{\sum x_i^2-n\bar{x}^2}\sum(x_i-\bar{x})E[y_i|x_i]$$ using the linearity of expectation. But by hypothesis, $$E[y_i|x_i]=E[\beta_0+\beta_1x_i+u|x_i]=\beta_0+\beta_1x_i$$ where we have used the linearity of expectation and the model assumption that $E[u|x_i]=0$. Plugging this back into our last formula, we have
$$E\left[\hat{\beta_1}\left|\right.x_i\right]=\frac{1}{\sum x_i^2-n\bar{x}^2}\sum(x_i-\bar{x})E[y_i|x_i]=\frac{\sum(x_i-\bar{x})(\beta_0+\beta_1x_i)}{\sum x_i^2-n\bar{x}^2}$$
A little algebra on the numerator gives $$\sum (x_i\beta_0+x_i^2\beta_1-\bar{x}\beta_0-\bar{x}x_i\beta_1)=\beta_0\underbrace{\sum x_i}_{=n\bar{x}}-\beta_0\underbrace{\sum\bar{x}}_{=n\bar{x}}+\beta_1\sum\left( x_i^2-x_i\bar{x}\right)$$
So the numerator simplifies to $$\beta_1\sum(x_i^2-x_i\bar{x})=\beta_1\left(\sum x_i^2-\bar{x}\underbrace{\sum x_i}_{=n\bar{x}}\right)=\beta_1\underbrace{\left(\sum x_i^2-n\bar{x}^2\right)}_{\text{the denominator of $E[\hat{\beta_1}|x_i]$}}$$
So we conclude $$\boxed{E\left[\hat{\beta_1}\left|\right.x_i\right]=\beta_1}$$
If you treat the independent variable $X$ as a random variable, then from you here you can use the law of iterated expectations to conclude $$\boxed{E\left[\hat{\beta_1}\right]=E\left[E\left[\hat{\beta_1}\left|\right.x_i\right]\right]=E[\beta_1]=\beta_1}$$ which is what unbiasedness means.
Note that the only model assumptions this proof depends on are
In particular, you don't need the assumption that the error $u$ is normally distributed. That distributional assumption is required only for inference (i.e. for deriving the sampling distributions of the estimators).