Show that the least squares estimator of the slope is an unbiased estimator of the `true' slope in the model. [closed]

Question

Under the assumptions of the classical simple linear regression model, show that the least squares estimator of the slope is an unbiased estimator of the `true' slope in the model.

Anyone have any ideas for the following questions?

Answer 1

The proof is mostly a matter of a little algebra.

The OLS (ordinary least squares) estimator for $\beta_1$ in the model $$y=\beta_0+\beta_1x+u$$ can be shown to have the form $$\hat{\beta_1}=\frac{\sum(x_i-\bar{x})y_i }{\sum x_i^2-n\bar{x}^2 }$$ Since you didn't say what you've tried, I don't know if you understand how to derive this expression from whatever your book defines $\hat{\beta_1}$ to be.

In any case, taking the conditional expectation with respect to the $x_i$, we get $$E\left[\hat{\beta_1}\left|\right.x_i\right]=\frac{1}{\sum x_i^2-n\bar{x}^2}\sum(x_i-\bar{x})E[y_i|x_i]$$ using the linearity of expectation. But by hypothesis, $$E[y_i|x_i]=E[\beta_0+\beta_1x_i+u|x_i]=\beta_0+\beta_1x_i$$ where we have used the linearity of expectation and the model assumption that $E[u|x_i]=0$. Plugging this back into our last formula, we have

$$E\left[\hat{\beta_1}\left|\right.x_i\right]=\frac{1}{\sum x_i^2-n\bar{x}^2}\sum(x_i-\bar{x})E[y_i|x_i]=\frac{\sum(x_i-\bar{x})(\beta_0+\beta_1x_i)}{\sum x_i^2-n\bar{x}^2}$$

A little algebra on the numerator gives $$\sum (x_i\beta_0+x_i^2\beta_1-\bar{x}\beta_0-\bar{x}x_i\beta_1)=\beta_0\underbrace{\sum x_i}_{=n\bar{x}}-\beta_0\underbrace{\sum\bar{x}}_{=n\bar{x}}+\beta_1\sum\left( x_i^2-x_i\bar{x}\right)$$

So the numerator simplifies to $$\beta_1\sum(x_i^2-x_i\bar{x})=\beta_1\left(\sum x_i^2-\bar{x}\underbrace{\sum x_i}_{=n\bar{x}}\right)=\beta_1\underbrace{\left(\sum x_i^2-n\bar{x}^2\right)}_{\text{the denominator of $E[\hat{\beta_1}|x_i]$}}$$

So we conclude $$\boxed{E\left[\hat{\beta_1}\left|\right.x_i\right]=\beta_1}$$

If you treat the independent variable $X$ as a random variable, then from you here you can use the law of iterated expectations to conclude $$\boxed{E\left[\hat{\beta_1}\right]=E\left[E\left[\hat{\beta_1}\left|\right.x_i\right]\right]=E[\beta_1]=\beta_1}$$ which is what unbiasedness means.

Note that the only model assumptions this proof depends on are

1. the conditional expectation of $y$ with respect to $x$ is $\beta_0+\beta_1x$
2. the conditional expectation of $u$ with respect to $x$ is zero
3. the data points $(x_i,y_i)$ can be thought of as an i.i.d. (independent and identically distributed) random sample from the "true" joint distribution of $x$ and $y$ (actually, you don't need independence, only the identically distributed part)

In particular, you don't need the assumption that the error $u$ is normally distributed. That distributional assumption is required only for inference (i.e. for deriving the sampling distributions of the estimators).