Consider the equation $y_i=\beta_0+\beta_1x_i+\epsilon_i$ for $i=1, \dotsc, n$.

We have unbiased estimators $b_0$ and $b_1$ for $\beta_0$ and $\beta_1$ respectively, where $b_0=\bar{y}-b_1\bar{x}$ and $b_1= S_{xy} / S_{xx}$.

How does one show that $\operatorname{Cov}(b_0,b_1)=-\frac{\sigma\bar{x}}{S_{xx}}$

I tried using $\operatorname{Cov}(b_0,b_1)=E(b_0b_1)-E(b_0)E(b_1)$ to no avail as it just equals $0$ when I try and do that. Thanks!


You have not defined your symbols, so presumably $$S_{xx}=\sum (x_i-\bar x)^2 \quad\text{ and }\quad S_{xy}=\sum (x_i-\bar x)(y_i-\bar y)$$

Use the bilinearity of covariance.

We have

\begin{align} \operatorname{Cov}(b_0,b_1)&=\operatorname{Cov}(\bar y-b_1\bar x,b_1) \\&=\operatorname{Cov}(\bar y,b_1)-\bar x\operatorname{Var}(b_1) \end{align}

Recall that $\bar y$ and $b_1$ are independently distributed, so their covariance vanishes.

Moreover, the exact distribution of $b_1$ is $$b_1\sim\mathcal N\left(\beta_1,\frac{\sigma^2}{S_{xx}}\right)$$

Hence you finally get as the covariance between the least square estimates $$\operatorname{Cov}(b_0,b_1)=-\frac{\bar x\sigma^2}{S_{xx}}$$

  • $\begingroup$ Thanks so much! And sorry for not defining the sum of squares! $\endgroup$ – Programmer Sep 2 '18 at 13:07
  • $\begingroup$ This answer assumes the errors $\epsilon_i$ are i.i.d $N(0,\sigma^2)$. $\endgroup$ – StubbornAtom Sep 30 '19 at 18:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.