# Correlation Coefficient and Determination Coefficient

I'm new to linear regression and am trying to teach myself.

In my textbook there's a problem that asks "why is $$R^{2}$$ in the regression of $$Y$$ on $$X$$ equal to the square of the sample correlation between X and Y?"

I've been throwing my head against this for a while and I keep getting stuck because in the correlation coefficient there is a $$X$$ and $$\bar{X}$$ term, whilst in the $$R^{2}$$ term there is no such thing.

Can anyone provide a derivation as to why $$R^{2}$$ is the correlation coefficient squared?

Thanks!

• It might help if you define the terms in your question. What is the equation for $R^2$, in particular?
– user856
May 10, 2012 at 9:27
• If by $R^2$ you mean the "explained variance", then stats.SE might be a more suitable site for this question. See, for example, this question or this one for some ideas related to this. Jan 4, 2013 at 23:46

Suppose that we have $n$ observations $(x_1,y_1),\ldots,(x_n,y_n)$ from a simple linear regression $$Y_i=\alpha+\beta x_i+\varepsilon_i,$$ where $i=1,\ldots,n$. Let us denote $\hat y_i=\hat\alpha+\hat\beta x_i$ for $i=1,\ldots,n$, where $\hat\alpha$ and $\hat\beta$ are the ordinary least squares estimators of the parameters $\alpha$ and $\beta$. The coefficient of the determination $r^2$ is defined by $$r^2=\frac{\sum_{i=1}^n(\hat y_i-\bar y)^2}{\sum_{i=1}^n(y_i-\bar y)^2}.$$ Using the facts that $$\hat\beta=\frac{\sum_{i=1}^n(x_i-\bar x)(y_i-\bar y)}{\sum_{i=1}^n(x_i-\bar x)^2}$$ and $\hat\alpha=\bar y-\hat\beta\bar x$, we obtain \begin{align*} \sum_{i=1}^n(\hat y_i-\bar y)^2 &=\sum_{i=1}^n(\hat\alpha+\hat\beta x_i-\bar y)^2\\ &=\sum_{i=1}^n(\bar y-\hat\beta\bar x+\hat\beta x_i-\bar y)^2\\ &=\hat\beta^2\sum_{i=1}^n(x_i-\bar x)^2\\ &=\frac{[\sum_{i=1}^n(x_i-\bar x)(y_i-\bar y)]^2\sum_{i=1}^n(x_i-\bar x)^2}{[\sum_{i=1}^n(x_i-\bar x)^2]^2}\\ &=\frac{[\sum_{i=1}^n(x_i-\bar x)(y_i-\bar y)]^2}{\sum_{i=1}^n(x_i-\bar x)^2}. \end{align*} Hence, \begin{align*} r^2 &=\frac{\sum_{i=1}^n(\hat y_i-\bar y)^2}{\sum_{i=1}^n(y_i-\bar y)^2}\\ &=\frac{[\sum_{i=1}^n(x_i-\bar x)(y_i-\bar y)]^2}{\sum_{i=1}^n(x_i-\bar x)^2\sum_{i=1}^n(y_i-\bar y)^2}\\ &=\biggl(\frac{\sum_{i=1}^n(x_i-\bar x)(y_i-\bar y)}{\sqrt{\sum_{i=1}^n(x_i-\bar x)^2\sum_{i=1}^n(y_i-\bar y)^2}}\biggr)^2. \end{align*} This shows that the coefficient of determination of a simple linear regression is the square of the sample correlation coefficient of $(x_1,y_1),\ldots,(x_n,y_n)$.

• Could anyone explain the reason for the downvote?.. Sep 29, 2017 at 7:48
• very clear explanation. Jul 22, 2019 at 12:03

The complete proof of how to derive the coefficient of determination $R^{2}$ from the Squared Pearson Correlation Coefficient between the observed values $y_{i}$ and the fitted values $\hat{y}_{i}$ can be found under the following link:

http://economictheoryblog.wordpress.com/2014/11/05/proof/

In my eyes it should be pretty easy to understand, just follow the single steps.

There are many forms of the computation available online (such as the Wikipedia page on the correlation coefficient http://en.wikipedia.org/wiki/Pearson_product-moment_correlation_coefficient#Pearson.27s_correlation_and_least_squares_regression_analysis ) but note that this is a magical algebraic property of least squares linear regression, not linear regression in general.

• hmm what I don't understand about that is why the correlation coeff equation doesn't have an X term any more? Apr 10, 2012 at 5:49
• Which equation does not have an X term?
– zyx
Apr 10, 2012 at 6:03

There are different forms to express R2: Some expressions have (X-Xbar) squared in the numerator, while others express it just with the square of predicted ys. All forms are equivalent.

References: Dougherty; Gujarati; Wooldridge

The following answer assumes you are familiar with linear algebra and matrix factorization. The basic idea is to express both the Pearson correlation and $$R^2$$ in terms of vector of observations (and intercept).

Suppose you have made $$n$$ observations $$(x_i, y_i): i = 1, 2, \ldots, n$$, denote the column vector $$(y_1, y_2, \ldots, y_n)'$$ and the column vector $$(x_1, x_2, \ldots, x_n)'$$ by $$y$$ and $$x$$ respectively. Moreover, denote the $$n$$-long column vector of all ones by $$e$$. The design matrix $$X$$ thus can be written as $$X = \begin{bmatrix} e & x \end{bmatrix}$$. Let $$X = QR$$ be the QR decomposition of $$X$$, where $$Q = \begin{bmatrix} q_1 & q_2\end{bmatrix}$$ is column-wise orthogonal, i.e., $$q_1'q_1 = q_2'q_2 = 1$$ and $$q_1'q_2 = 0$$.

With these preparations, it is readily seen that the Pearson correlation $$r$$ is \begin{align} r = \frac{\sum(x_i - \bar{x})(y_i - \bar{y})}{\sqrt{\sum(x_i - \bar{x})^2\sum(y_i - \bar{y})^2}} = \frac{x'(I - q_1q_1')y}{\sqrt{x'(I - q_1q_1')x \cdot y'(I - q_1q_1')y}}. \tag{1} \end{align} While the coefficient of determination is given by \begin{align} R^2 = \frac{\sum(\hat{y}_i - \bar{y})^2}{\sum(y_i - \bar{y})^2} = \frac{y'(H - q_1q_1')y}{y'(I - q_1q_1')y} = \frac{(y'q_2)^2}{y'(I - q_1q_1')y}, \tag{2} \end{align} where $$H = X(X'X)^{-1}X' = QQ' = q_1q_1' + q_2q_2'$$ is the so-called hat matrix. It thus follows by $$(1)$$ and $$(2)$$ that to show $$R^2 = r^2$$, it is sufficient to prove \begin{align} (x'(I - q_1q_1')y)^2 = (y'q_2)^2 \times x'(I - q_1q_1')x. \tag{3} \end{align} Since $$x$$ is in the space spanned by two orthonormal vectors $$q_1$$ and $$q_2$$, we have $$x = (x'q_1)q_1 + (x'q_2)q_2$$, whence \begin{align} & x'(I - q_1q_1')x = \|x\|^2 - (x'q_1)^2 = (x'q_2)^2, \\ & x'(I - q_1q_1')y = ((x'q_1)q_1' + (x'q_2)q_2')(y - q_1q_1'y) = (x'q_2)q_2'y. \end{align} Hence $$(3)$$ holds. This completes the proof.