0
$\begingroup$

According to Wiki:

https://en.wikipedia.org/wiki/Fraction_of_variance_unexplained

$1 - R^2 = VAR_{err}/VAR_{tot}$

Where $VAR_{err} = \sum_{i = 1}^N (y_i - \hat{y}_i)^2$ is the variance of the residuals.

I don't see how this is correct, unless the residuals have mean zero. To see this, define the residuals:

$e_i = y_i - \hat{y}_i$

The variance of the residuals is:

$\sum_{i = 1}^N (e_i - \bar{e}_i)^2 = \sum_{i = 1}^N (y_i - \hat{y}_i - \bar{e}_i)^2 $

Do you disagree or is Wikipedia wrong?

$\endgroup$
0
$\begingroup$

Note that in OLS regression with an intercept term, due to the "first order condition", when you take derivative w.r.t. $\beta_0$ you have $$ -2\sum_{i=1}^n (y_i - \hat{\beta}_0 - \hat{\beta}_1x_{1i} - \cdots \hat{\beta}_px_{pi}) = -2\sum e_i =0, $$ hence $$ \bar{e}_n = 0, $$ thus $$ \hat{\sigma}^2 = \frac{1}{n}\sum(e_i - \bar{e})^2 = \frac{\sum e_i ^ 2}{n}. $$

$\endgroup$
  • $\begingroup$ Thanks for the answer, but this not only hold for linear models? I think Wiki is wrong since it treats the more general case, but still imposes this assumption. $\endgroup$ – hans Dec 25 '18 at 9:13
  • $\begingroup$ @hans The residuals have zero mean for any model with an intercept term and least-squares estimation. $\endgroup$ – V. Vancak Dec 25 '18 at 9:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.