# Show that the least squares estimator of the slope is an unbiased estimator of the true' slope in the model. [closed]

Under the assumptions of the classical simple linear regression model, show that the least squares estimator of the slope is an unbiased estimator of the true' slope in the model.

Anyone have any ideas for the following questions?

The proof is mostly a matter of a little algebra.

The OLS (ordinary least squares) estimator for $\beta_1$ in the model $$y=\beta_0+\beta_1x+u$$ can be shown to have the form $$\hat{\beta_1}=\frac{\sum(x_i-\bar{x})y_i }{\sum x_i^2-n\bar{x}^2 }$$ Since you didn't say what you've tried, I don't know if you understand how to derive this expression from whatever your book defines $\hat{\beta_1}$ to be.

In any case, taking the conditional expectation with respect to the $x_i$, we get $$E\left[\hat{\beta_1}\left|\right.x_i\right]=\frac{1}{\sum x_i^2-n\bar{x}^2}\sum(x_i-\bar{x})E[y_i|x_i]$$ using the linearity of expectation. But by hypothesis, $$E[y_i|x_i]=E[\beta_0+\beta_1x_i+u|x_i]=\beta_0+\beta_1x_i$$ where we have used the linearity of expectation and the model assumption that $E[u|x_i]=0$. Plugging this back into our last formula, we have

$$E\left[\hat{\beta_1}\left|\right.x_i\right]=\frac{1}{\sum x_i^2-n\bar{x}^2}\sum(x_i-\bar{x})E[y_i|x_i]=\frac{\sum(x_i-\bar{x})(\beta_0+\beta_1x_i)}{\sum x_i^2-n\bar{x}^2}$$

A little algebra on the numerator gives $$\sum (x_i\beta_0+x_i^2\beta_1-\bar{x}\beta_0-\bar{x}x_i\beta_1)=\beta_0\underbrace{\sum x_i}_{=n\bar{x}}-\beta_0\underbrace{\sum\bar{x}}_{=n\bar{x}}+\beta_1\sum\left( x_i^2-x_i\bar{x}\right)$$

So the numerator simplifies to $$\beta_1\sum(x_i^2-x_i\bar{x})=\beta_1\left(\sum x_i^2-\bar{x}\underbrace{\sum x_i}_{=n\bar{x}}\right)=\beta_1\underbrace{\left(\sum x_i^2-n\bar{x}^2\right)}_{\text{the denominator of E[\hat{\beta_1}|x_i]}}$$

So we conclude $$\boxed{E\left[\hat{\beta_1}\left|\right.x_i\right]=\beta_1}$$

If you treat the independent variable $X$ as a random variable, then from you here you can use the law of iterated expectations to conclude $$\boxed{E\left[\hat{\beta_1}\right]=E\left[E\left[\hat{\beta_1}\left|\right.x_i\right]\right]=E[\beta_1]=\beta_1}$$ which is what unbiasedness means.

Note that the only model assumptions this proof depends on are

1. the conditional expectation of $y$ with respect to $x$ is $\beta_0+\beta_1x$
2. the conditional expectation of $u$ with respect to $x$ is zero
3. the data points $(x_i,y_i)$ can be thought of as an i.i.d. (independent and identically distributed) random sample from the "true" joint distribution of $x$ and $y$ (actually, you don't need independence, only the identically distributed part)

In particular, you don't need the assumption that the error $u$ is normally distributed. That distributional assumption is required only for inference (i.e. for deriving the sampling distributions of the estimators).

• How do you go from The OLS (ordinary least squares) estimator for $\beta_1$ in the model $$y=\beta_0+\beta_1x+u$$ can be shown to have the form $$\hat{\beta_1}=\frac{\sum(x_i-\bar{x})y_i }{\sum x_i^2-n\bar{x}^2 }$$? Since you just show $$\hat{\beta_1}=\frac{\sum(x_i-\bar{x})y_i }{\sum x_i^2-n\bar{x}^2 }$$, how can you just assert it? May 9, 2014 at 21:24
• You are confusing several things. (1) There is a difference between the population parameter $\beta_1$ and the estimator $\hat{\beta_1}$. (2) I did not "show" that $\hat{\beta_1}$ has the form I claimed it has; I said it "can be shown." The OLS estimator $\hat{\beta_1}$ has several different but equivalent forms. One is $r\frac{s_y}{s_x}$, where $r$ is the sample correlation between $x$ and $y$, and $s_y$ and $s_x$ are the sample standard deviations. A little algebra shows this is equivalent to the formula I used. May 9, 2014 at 23:17
• This is what I got... I still do not get what you have though?$$\hat{\beta_1} = \frac{S_{xy}}{S_{xx}} = \frac{\sum_{i=1}^{n} (x_i - \bar{x})(Y_i - \bar{Y})}{\sum_{i=1}^n (x_i - \bar{x})^2} = \frac{\sum_{i=1}^{n} (x_i - \bar{x})Y_i -\sum_{i=1}^{n} (x_i - \bar{x})\bar{Y}}{\sum_{i=1}^n (x_i - \bar{x})^2}$$ May 9, 2014 at 23:42
• $\sum(x_i-\bar{x})\bar{y}=\bar{y}\sum(x_i-\bar{x})=\bar{y}\cdot 0=0$ and $\sum(x_i-\bar{x})^2=\sum(x_i^2-2x_i\bar{x}+\bar{x}^2)=(\sum x_i^2)-2\bar{x}(\sum x_i)+n\bar{x}^2=(\sum x_i^2)-2\bar{x}(n\bar{x})+n\bar{x}^2=(\sum x_i^2)-n\bar{x}^2$. May 9, 2014 at 23:56
• No... You can't factor $y_i$ out of the sum. May 10, 2014 at 1:10