1
$\begingroup$

In the book "Introduction to Statistical Learning" page 66, there are formulas of the standard errors of the coefficient estimates $\hat{\beta}_0$ and $\hat{\beta}_1$. I know the proof of $SE(\hat{\beta}_1)$ but I am confused about how to derive the formula for $$SE(\hat{\beta}_0)^2 = \sigma^2\left[\frac{1}{n} + \frac{\bar{x}^2}{{\sum_{i=1}^n}(x_i - \bar{x})^2}\right]$$ since $\sigma^2 = Var(\epsilon)$, not the variance of $y_i's$.

My calculation so far is as follows: $$Var(\hat{\beta}_0) = Var(\bar{y} - \hat{\beta}_1\bar{x}) = Var(\bar{y}) + \bar{x}^2\frac{\sigma^2}{{\sum_{i=1}^n}(x_i - \bar{x})^2} - 2\bar{x} Cov(\bar{y}, \hat{\beta}_1) $$in which $\sigma^2 = Var(\epsilon)$.

$Cov(\bar{y}, \hat{\beta}_1) = 0$ since $\bar{y}$ and $\hat{\beta}_1$ are uncorrelated. $Var(\bar{y}) = \frac{\sigma^2}{n}$ in which $\sigma^2 = Var(y_i)$.

So how can we have the formula for $SE(\hat{\beta}_0)^2$ as above since the 2 $\sigma's$ are different from each other?

Many thanks in advance for your help!

$\endgroup$
8
  • $\begingroup$ If you know that $Var( \hat \beta_1)=\frac{\sigma^2}{{\sum_{i=1}^n}(x_i - \bar{x})^2}$ and you use $Var(\hat \beta_1\cdot \overline x)=\overline x^2\cdot Var(\hat \beta_1)$ the formula for $Var( \hat \beta_0)$ follows straightforward. I don´t see any difference. $\endgroup$ Oct 15, 2018 at 19:16
  • $\begingroup$ But they are the same $Var(y_i) =Var(\beta_0 + \beta_1 x_i +\epsilon_i) =0 + Var(\epsilon_i)$ $\endgroup$ Oct 15, 2018 at 19:29
  • $\begingroup$ @callculus But $SE(\hat{\beta_0})^2 = \sigma^2[\frac{1}{n} + \frac{\bar{x}^2}{{\sum_{i=1}^n}(x_i - \bar{x})^2}]$, not just $\overline x^2\cdot Var(\hat \beta_1)$. There is also a component of $\sigma^2\frac{1}{n}$. $\endgroup$
    – Sophil
    Oct 15, 2018 at 21:08
  • 1
    $\begingroup$ In linear regression, explanatory variables $X$ (or independent variables) are not random variables, neither (the real value of) $\beta_0$ and $\beta_1$ . $\endgroup$ Oct 16, 2018 at 6:35
  • 1
    $\begingroup$ In the linear regression $y_1,y_2,\cdots,y_n $ are independent with $y_i\sim N(\beta_0+\beta_1 x_i,\sigma^2)$ so $\overline{y}\sim N(\beta_0+\beta_1\bar x,\sigma^2/n)$, without the use of CLT. But in many cases when n is big we can assume Normal distribution due to CLT. $\endgroup$ Oct 16, 2018 at 7:26

1 Answer 1

0
$\begingroup$

You have found out that

$$Var(\hat{\beta}_0) = Var(\bar{y} - \hat{\beta}_1\bar{x}) = Var(\bar{y}) + \bar{x}^2\frac{\sigma^2}{{\sum_{i=1}^n}(x_i - \bar{x})^2} - 2\bar{x} \underbrace{Cov(\bar{y}, \hat{\beta}_1)}_{=0}$$ in which $Var(\bar{y}) =\frac{\sigma^2}{n}$.

Inserting $\frac{\sigma^2}{n}$ for $Var(\bar{y}) $

$$Var(\hat{\beta}_0) = \frac{\sigma^2}{n} + \bar{x}^2\frac{\sigma^2}{{\sum_{i=1}^n}(x_i - \bar{x})^2}=\sigma^2 \cdot \left(\frac{1}{n} + \bar{x}^2\frac{1}{{\sum_{i=1}^n}(x_i - \bar{x})^2} \right)$$

$\endgroup$
1
  • $\begingroup$ Thanks for your answer! Actually, my question is how we can put the common factor $\sigma^2$ while one is the variance of the error terms $\epsilon$ (in the formula for $\hat{\beta}_1$) and the other in $Var(\bar{y)}$ is the variance of $y_i's$. Maybe I wasn't clear in my question. My concern has been answered by papasmurfete, which is $Var(\bar{y}) = Var(\epsilon)$. $\endgroup$
    – Sophil
    Oct 16, 2018 at 6:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.