12
$\begingroup$

I'm really new to linear regression and am trying to teach myself.

In my textbook there's a problem that asks why $R^{2}$ in the regression of $Y$ on $X =$ the sample correlation between X and Y the whole squared.

I've been throwing my head against this for a while and I keep getting stuck because in the correlation coefficient there is a $X$ and $\bar{X}$ term, whilst in the $R^{2}$ term there is no such thing.

Can anyone provide a derivation as to why $R^{2}$ is the correlation coefficient squared?

Thanks!

$\endgroup$
  • 3
    $\begingroup$ It might help if you define the terms in your question. What is the equation for $R^2$, in particular? $\endgroup$ – Rahul May 10 '12 at 9:27
  • 1
    $\begingroup$ If by $R^2$ you mean the "explained variance", then stats.SE might be a more suitable site for this question. See, for example, this question or this one for some ideas related to this. $\endgroup$ – Dilip Sarwate Jan 4 '13 at 23:46
11
$\begingroup$

Suppose that we have $n$ observations $(x_1,y_1),\ldots,(x_n,y_n)$ from a simple linear regression $$ Y_i=\alpha+\beta x_i+\varepsilon_i, $$ where $i=1,\ldots,n$. Let us denote $\hat y_i=\hat\alpha+\hat\beta x_i$ for $i=1,\ldots,n$, where $\hat\alpha$ and $\hat\beta$ are the ordinary least squares estimators of the parameters $\alpha$ and $\beta$. The coefficient of the determination $r^2$ is defined by $$ r^2=\frac{\sum_{i=1}^n(\hat y_i-\bar y)^2}{\sum_{i=1}^n(y_i-\bar y)^2}. $$ Using the facts that $$ \hat\beta=\frac{\sum_{i=1}^n(x_i-\bar x)(y_i-\bar y)}{\sum_{i=1}^n(x_i-\bar x)^2} $$ and $\hat\alpha=\bar y-\hat\beta\bar x$, we obtain \begin{align*} \sum_{i=1}^n(\hat y_i-\bar y)^2 &=\sum_{i=1}^n(\hat\alpha+\hat\beta x_i-\bar y)^2\\ &=\sum_{i=1}^n(\bar y-\hat\beta\bar x+\hat\beta x_i-\bar y)^2\\ &=\hat\beta^2\sum_{i=1}^n(x_i-\bar x)^2\\ &=\frac{[\sum_{i=1}^n(x_i-\bar x)(y_i-\bar y)]^2\sum_{i=1}^n(x_i-\bar x)^2}{[\sum_{i=1}^n(x_i-\bar x)^2]^2}\\ &=\frac{[\sum_{i=1}^n(x_i-\bar x)(y_i-\bar y)]^2}{\sum_{i=1}^n(x_i-\bar x)^2}. \end{align*} Hence, \begin{align*} r^2 &=\frac{\sum_{i=1}^n(\hat y_i-\bar y)^2}{\sum_{i=1}^n(y_i-\bar y)^2}\\ &=\frac{[\sum_{i=1}^n(x_i-\bar x)(y_i-\bar y)]^2}{\sum_{i=1}^n(x_i-\bar x)^2\sum_{i=1}^n(y_i-\bar y)^2}\\ &=\biggl(\frac{\sum_{i=1}^n(x_i-\bar x)(y_i-\bar y)}{\sqrt{\sum_{i=1}^n(x_i-\bar x)^2\sum_{i=1}^n(y_i-\bar y)^2}}\biggr)^2. \end{align*} This shows that the coefficient of determination of a simple linear regression is the square of the sample correlation coefficient of $(x_1,y_1),\ldots,(x_n,y_n)$.

$\endgroup$
  • 3
    $\begingroup$ Could anyone explain the reason for the downvote?.. $\endgroup$ – Cm7F7Bb Sep 29 '17 at 7:48
8
$\begingroup$

The complete proof of how to derive the coefficient of determination $R^{2}$ from the Squared Pearson Correlation Coefficient between the observed values $y_{i}$ and the fitted values $\hat{y}_{i}$ can be found under the following link:

http://economictheoryblog.wordpress.com/2014/11/05/proof/

In my eyes it should be pretty easy to understand, just follow the single steps.

$\endgroup$
1
$\begingroup$

There are many forms of the computation available online (such as the Wikipedia page on the correlation coefficient http://en.wikipedia.org/wiki/Pearson_product-moment_correlation_coefficient#Pearson.27s_correlation_and_least_squares_regression_analysis ) but note that this is a magical algebraic property of least squares linear regression, not linear regression in general.

$\endgroup$
  • $\begingroup$ hmm what I don't understand about that is why the correlation coeff equation doesn't have an X term any more? $\endgroup$ – Scubadiver Apr 10 '12 at 5:49
  • $\begingroup$ Which equation does not have an X term? $\endgroup$ – zyx Apr 10 '12 at 6:03
0
$\begingroup$

There are different forms to express R2: Some expressions have (X-Xbar) squared in the numerator, while others express it just with the square of predicted ys. All forms are equivalent.

References: Dougherty; Gujarati; Wooldridge

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.