# How can I prove the variance of residuals in simple linear regression?

How can I prove the variance of residuals in simple linear regression?

$\operatorname{var}(r_i)=\sigma^2\left[1-\frac{1}{n}-\dfrac{(x_i-\bar{x})^2}{\sum_{l=1}^{n}(x_l-\bar{x})}\right]$

I tried..

using $r_i=y_i-\hat{y_i}$

$\operatorname{var}(r_i)=\operatorname{var}(y_i-\hat{y_i})=\operatorname{var}(y_i-\bar{y})+\operatorname{var}(\hat{\beta_1}(x_i-\bar{x}))-2\operatorname{Cov}((y_i-\bar{y}),\hat{\beta_1}(x_i-\bar{x}))$

How can I go further?

• Your question seems incomplete. Jun 3, 2013 at 18:00
• modified. thanks. Jun 3, 2013 at 18:09
• Sep 25, 2019 at 13:50
• Your first expression seems to be missing a $\,^2$ in the denominator Apr 18, 2020 at 1:41

I believe the previous answer posted is incorrect, since $y_i$ and $\hat y_i$ are not uncorrelated. I would prove this as follows:

\begin{align} \text{Cov}(r) &= \text{Cov}(y - Py), \quad P = X(X^TX)^{-1}X^T \\ & = \text{Cov}((I_n-P)y) \\ & = (I_n - P)\ \text{Cov}(y)\ (I_n - P)^T \\ & = (I_n-P)\ \sigma^2 I_n\ (I_n - P)^T \end{align}

from which we can conclude that $\text{var}(r_i)=\sigma^2 (1 - P_{ii})$. It should be quite simple to confirm that your equation is recovered when you let $X$ be the matrix with a column of $1$'s (to represent $\bar x$) and a second column of the $x_i$'s.

• This is not simple linear regression anymore since you are using vectors rather than scalars. Oct 1, 2014 at 7:06
• @Will, that is why I said "let X be the matrix with a column of 1's (to represent x¯) and a second column of the xi's." Oct 1, 2014 at 11:04

Note that \begin{align}\operatorname{Var}(r_i) &=\operatorname{Var}(y_i-\hat{y_i}) \\ &=\operatorname{Var}(y_i)+ \operatorname{Var}(\hat{y_i}) \\ &= \sigma^2 + \operatorname{Var}(\bar y+\hat \beta _1(x_i-\bar x))\\ &= \sigma^2 +\operatorname{Var}(\bar y)+(x_i-\bar x)^2 \cdot\operatorname{Var}(\hat \beta _1)\\&=\sigma^2 +\dfrac{\sigma^2}{n}+\dfrac{\sigma^2 \cdot (x_i-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}\\ &=\sigma^2 \left[1+\dfrac 1n +\dfrac{(x_i-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2} \right]\end{align}

• This incorrectly assumes that $y$ and $\hat{y}$ are uncorrelated. That this is not generally the case should be obvious (if they were uncorrelated, the slope would be 0). Sep 11, 2014 at 0:39
• Not only is the proof incorrect -- the formula you have derived is not correct and doesn't match the formula in the question. Terms 2 and 3 should be negative, not positive. Jan 26, 2020 at 19:17