# Show that the least squares estimator of the slope is an unbiased estimator of the true' slope in the model. [closed]

Under the assumptions of the classical simple linear regression model, show that the least squares estimator of the slope is an unbiased estimator of the true' slope in the model.

Anyone have any ideas for the following questions?

The proof is mostly a matter of a little algebra.

The OLS (ordinary least squares) estimator for $\beta_1$ in the model $$y=\beta_0+\beta_1x+u$$ can be shown to have the form $$\hat{\beta_1}=\frac{\sum(x_i-\bar{x})y_i }{\sum x_i^2-n\bar{x}^2 }$$ Since you didn't say what you've tried, I don't know if you understand how to derive this expression from whatever your book defines $\hat{\beta_1}$ to be.

In any case, taking the conditional expectation with respect to the $x_i$, we get $$E\left[\hat{\beta_1}\left|\right.x_i\right]=\frac{1}{\sum x_i^2-n\bar{x}^2}\sum(x_i-\bar{x})E[y_i|x_i]$$ using the linearity of expectation. But by hypothesis, $$E[y_i|x_i]=E[\beta_0+\beta_1x_i+u|x_i]=\beta_0+\beta_1x_i$$ where we have used the linearity of expectation and the model assumption that $E[u|x_i]=0$. Plugging this back into our last formula, we have

$$E\left[\hat{\beta_1}\left|\right.x_i\right]=\frac{1}{\sum x_i^2-n\bar{x}^2}\sum(x_i-\bar{x})E[y_i|x_i]=\frac{\sum(x_i-\bar{x})(\beta_0+\beta_1x_i)}{\sum x_i^2-n\bar{x}^2}$$

A little algebra on the numerator gives $$\sum (x_i\beta_0+x_i^2\beta_1-\bar{x}\beta_0-\bar{x}x_i\beta_1)=\beta_0\underbrace{\sum x_i}_{=n\bar{x}}-\beta_0\underbrace{\sum\bar{x}}_{=n\bar{x}}+\beta_1\sum\left( x_i^2-x_i\bar{x}\right)$$

So the numerator simplifies to $$\beta_1\sum(x_i^2-x_i\bar{x})=\beta_1\left(\sum x_i^2-\bar{x}\underbrace{\sum x_i}_{=n\bar{x}}\right)=\beta_1\underbrace{\left(\sum x_i^2-n\bar{x}^2\right)}_{\text{the denominator of E[\hat{\beta_1}|x_i]}}$$

So we conclude $$\boxed{E\left[\hat{\beta_1}\left|\right.x_i\right]=\beta_1}$$

If you treat the independent variable $X$ as a random variable, then from you here you can use the law of iterated expectations to conclude $$\boxed{E\left[\hat{\beta_1}\right]=E\left[E\left[\hat{\beta_1}\left|\right.x_i\right]\right]=E[\beta_1]=\beta_1}$$ which is what unbiasedness means.

Note that the only model assumptions this proof depends on are

1. the conditional expectation of $y$ with respect to $x$ is $\beta_0+\beta_1x$
2. the conditional expectation of $u$ with respect to $x$ is zero
3. the data points $(x_i,y_i)$ can be thought of as an i.i.d. (independent and identically distributed) random sample from the "true" joint distribution of $x$ and $y$ (actually, you don't need independence, only the identically distributed part)

In particular, you don't need the assumption that the error $u$ is normally distributed. That distributional assumption is required only for inference (i.e. for deriving the sampling distributions of the estimators).

• How do you go from The OLS (ordinary least squares) estimator for $\beta_1$ in the model $$y=\beta_0+\beta_1x+u$$ can be shown to have the form $$\hat{\beta_1}=\frac{\sum(x_i-\bar{x})y_i }{\sum x_i^2-n\bar{x}^2 }$$? Since you just show $$\hat{\beta_1}=\frac{\sum(x_i-\bar{x})y_i }{\sum x_i^2-n\bar{x}^2 }$$, how can you just assert it? – afsdf dfsaf May 9 '14 at 21:24
• You are confusing several things. (1) There is a difference between the population parameter $\beta_1$ and the estimator $\hat{\beta_1}$. (2) I did not "show" that $\hat{\beta_1}$ has the form I claimed it has; I said it "can be shown." The OLS estimator $\hat{\beta_1}$ has several different but equivalent forms. One is $r\frac{s_y}{s_x}$, where $r$ is the sample correlation between $x$ and $y$, and $s_y$ and $s_x$ are the sample standard deviations. A little algebra shows this is equivalent to the formula I used. – symplectomorphic May 9 '14 at 23:17
• This is what I got... I still do not get what you have though?$$\hat{\beta_1} = \frac{S_{xy}}{S_{xx}} = \frac{\sum_{i=1}^{n} (x_i - \bar{x})(Y_i - \bar{Y})}{\sum_{i=1}^n (x_i - \bar{x})^2} = \frac{\sum_{i=1}^{n} (x_i - \bar{x})Y_i -\sum_{i=1}^{n} (x_i - \bar{x})\bar{Y}}{\sum_{i=1}^n (x_i - \bar{x})^2}$$ – afsdf dfsaf May 9 '14 at 23:42
• $\sum(x_i-\bar{x})\bar{y}=\bar{y}\sum(x_i-\bar{x})=\bar{y}\cdot 0=0$ and $\sum(x_i-\bar{x})^2=\sum(x_i^2-2x_i\bar{x}+\bar{x}^2)=(\sum x_i^2)-2\bar{x}(\sum x_i)+n\bar{x}^2=(\sum x_i^2)-2\bar{x}(n\bar{x})+n\bar{x}^2=(\sum x_i^2)-n\bar{x}^2$. – symplectomorphic May 9 '14 at 23:56
• No... You can't factor $y_i$ out of the sum. – symplectomorphic May 10 '14 at 1:10