# Standard Error of Coefficients in simple Linear Regression

In the book "Introduction to Statistical Learning" page 66, there are formulas of the standard errors of the coefficient estimates $$\hat{\beta}_0$$ and $$\hat{\beta}_1$$. I know the proof of $$SE(\hat{\beta}_1)$$ but I am confused about how to derive the formula for $$SE(\hat{\beta}_0)^2 = \sigma^2\left[\frac{1}{n} + \frac{\bar{x}^2}{{\sum_{i=1}^n}(x_i - \bar{x})^2}\right]$$ since $$\sigma^2 = Var(\epsilon)$$, not the variance of $$y_i's$$.

My calculation so far is as follows: $$Var(\hat{\beta}_0) = Var(\bar{y} - \hat{\beta}_1\bar{x}) = Var(\bar{y}) + \bar{x}^2\frac{\sigma^2}{{\sum_{i=1}^n}(x_i - \bar{x})^2} - 2\bar{x} Cov(\bar{y}, \hat{\beta}_1)$$in which $$\sigma^2 = Var(\epsilon)$$.

$$Cov(\bar{y}, \hat{\beta}_1) = 0$$ since $$\bar{y}$$ and $$\hat{\beta}_1$$ are uncorrelated. $$Var(\bar{y}) = \frac{\sigma^2}{n}$$ in which $$\sigma^2 = Var(y_i)$$.

So how can we have the formula for $$SE(\hat{\beta}_0)^2$$ as above since the 2 $$\sigma's$$ are different from each other?

Many thanks in advance for your help!

• If you know that $Var( \hat \beta_1)=\frac{\sigma^2}{{\sum_{i=1}^n}(x_i - \bar{x})^2}$ and you use $Var(\hat \beta_1\cdot \overline x)=\overline x^2\cdot Var(\hat \beta_1)$ the formula for $Var( \hat \beta_0)$ follows straightforward. I don´t see any difference. – callculus Oct 15 '18 at 19:16
• But they are the same $Var(y_i) =Var(\beta_0 + \beta_1 x_i +\epsilon_i) =0 + Var(\epsilon_i)$ – papasmurfete Oct 15 '18 at 19:29
• @callculus But $SE(\hat{\beta_0})^2 = \sigma^2[\frac{1}{n} + \frac{\bar{x}^2}{{\sum_{i=1}^n}(x_i - \bar{x})^2}]$, not just $\overline x^2\cdot Var(\hat \beta_1)$. There is also a component of $\sigma^2\frac{1}{n}$. – Sophil Oct 15 '18 at 21:08
• In linear regression, explanatory variables $X$ (or independent variables) are not random variables, neither (the real value of) $\beta_0$ and $\beta_1$ . – papasmurfete Oct 16 '18 at 6:35
• In the linear regression $y_1,y_2,\cdots,y_n$ are independent with $y_i\sim N(\beta_0+\beta_1 x_i,\sigma^2)$ so $\overline{y}\sim N(\beta_0+\beta_1\bar x,\sigma^2/n)$, without the use of CLT. But in many cases when n is big we can assume Normal distribution due to CLT. – papasmurfete Oct 16 '18 at 7:26

## 1 Answer

You have found out that

$$Var(\hat{\beta}_0) = Var(\bar{y} - \hat{\beta}_1\bar{x}) = Var(\bar{y}) + \bar{x}^2\frac{\sigma^2}{{\sum_{i=1}^n}(x_i - \bar{x})^2} - 2\bar{x} \underbrace{Cov(\bar{y}, \hat{\beta}_1)}_{=0}$$ in which $$Var(\bar{y}) =\frac{\sigma^2}{n}$$.

Inserting $$\frac{\sigma^2}{n}$$ for $$Var(\bar{y})$$

$$Var(\hat{\beta}_0) = \frac{\sigma^2}{n} + \bar{x}^2\frac{\sigma^2}{{\sum_{i=1}^n}(x_i - \bar{x})^2}=\sigma^2 \cdot \left(\frac{1}{n} + \bar{x}^2\frac{1}{{\sum_{i=1}^n}(x_i - \bar{x})^2} \right)$$

• Thanks for your answer! Actually, my question is how we can put the common factor $\sigma^2$ while one is the variance of the error terms $\epsilon$ (in the formula for $\hat{\beta}_1$) and the other in $Var(\bar{y)}$ is the variance of $y_i's$. Maybe I wasn't clear in my question. My concern has been answered by papasmurfete, which is $Var(\bar{y}) = Var(\epsilon)$. – Sophil Oct 16 '18 at 6:44