2
$\begingroup$

How can I prove the variance of residuals in simple linear regression?

Please help me.

$ \operatorname{var}(r_i)=\sigma^2\left[1-\frac{1}{n}-\dfrac{(x_i-\bar{x})^2}{\sum_{l=1}^{n}(x_l-\bar{x})}\right]$

I tried..

using $r_i=y_i-\hat{y_i}$

$\operatorname{var}(r_i)=\operatorname{var}(y_i-\hat{y_i})=\operatorname{var}(y_i-\bar{y})+\operatorname{var}(\hat{\beta_1}(x_i-\bar{x}))-2\operatorname{Cov}((y_i-\bar{y}),\hat{\beta_1}(x_i-\bar{x}))$

How can I go further?

If there's more information needed, please ask me to provide it.

$\endgroup$
4
  • $\begingroup$ Your question seems incomplete. $\endgroup$
    – response
    Jun 3, 2013 at 18:00
  • $\begingroup$ modified. thanks. $\endgroup$ Jun 3, 2013 at 18:09
  • 1
    $\begingroup$ stats.stackexchange.com/questions/115011/…. $\endgroup$ Sep 25, 2019 at 13:50
  • $\begingroup$ Your first expression seems to be missing a $\,^2$ in the denominator $\endgroup$
    – Henry
    Apr 18, 2020 at 1:41

2 Answers 2

2
$\begingroup$

I believe the previous answer posted is incorrect, since $y_i$ and $\hat y_i$ are not uncorrelated. I would prove this as follows:

$\begin{align} \text{Cov}(r) &= \text{Cov}(y - Py), \quad P = X(X^TX)^{-1}X^T \\ & = \text{Cov}((I_n-P)y) \\ & = (I_n - P)\ \text{Cov}(y)\ (I_n - P)^T \\ & = (I_n-P)\ \sigma^2 I_n\ (I_n - P)^T \end{align}$

from which we can conclude that $\text{var}(r_i)=\sigma^2 (1 - P_{ii})$. It should be quite simple to confirm that your equation is recovered when you let $X$ be the matrix with a column of $1$'s (to represent $\bar x$) and a second column of the $x_i$'s.

$\endgroup$
2
  • $\begingroup$ This is not simple linear regression anymore since you are using vectors rather than scalars. $\endgroup$ Oct 1, 2014 at 7:06
  • $\begingroup$ @Will, that is why I said "let X be the matrix with a column of 1's (to represent x¯) and a second column of the xi's." $\endgroup$
    – Brendon
    Oct 1, 2014 at 11:04
0
$\begingroup$

Note that $$\begin{align}\operatorname{Var}(r_i) &=\operatorname{Var}(y_i-\hat{y_i}) \\ &=\operatorname{Var}(y_i)+ \operatorname{Var}(\hat{y_i}) \\ &= \sigma^2 + \operatorname{Var}(\bar y+\hat \beta _1(x_i-\bar x))\\ &= \sigma^2 +\operatorname{Var}(\bar y)+(x_i-\bar x)^2 \cdot\operatorname{Var}(\hat \beta _1)\\&=\sigma^2 +\dfrac{\sigma^2}{n}+\dfrac{\sigma^2 \cdot (x_i-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}\\ &=\sigma^2 \left[1+\dfrac 1n +\dfrac{(x_i-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2} \right]\end{align}$$

$\endgroup$
2
  • 4
    $\begingroup$ This incorrectly assumes that $y$ and $\hat{y}$ are uncorrelated. That this is not generally the case should be obvious (if they were uncorrelated, the slope would be 0). $\endgroup$
    – Glen_b
    Sep 11, 2014 at 0:39
  • $\begingroup$ Not only is the proof incorrect -- the formula you have derived is not correct and doesn't match the formula in the question. Terms 2 and 3 should be negative, not positive. $\endgroup$
    – Denziloe
    Jan 26, 2020 at 19:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.