# Correlation between variables

I asked this question on stats SE but did not find a suitable answer so far. Maybe someone can help.

Given n random variables x1,...,xn (one-dimensional). The following is known (corr() = Pearson correlation):

corr(x1,x2) = a
corr(x2,x3) = a


The actual values of the random variables and their covariances are unkown though. Only some of their correlations are known.

From this, is it possible to calculate

corr(x3,x1) = ?


or give an estimate of the lowest possible correlation coefficient

corr(x3,x1) > a


More generally:

Given set of correlations

corr(x_i, x_i+1) with i=[1..c], c<n


is it possible to either directly calculate

corr(x_1, x_c+1)


or give a lower bound a of the coefficient with

corr(x_1, x_c+1) > a


I find it most intuitive to use the cholesky-decomposition of some correlation-matrix to look at such questions. The cholesky-decomposition provides a lower triangular matrix which always has (given the variables $\small x_1,x_2,x_3$) the form
$\qquad \small \begin{array} {r|lll} x_1: & 1 & . & . & \\ x_2: & a_1 & a_2 & . \\ x_3: & b_1 & b_2 & b_3 \\ \end{array}$
which can be continued to more rows/columns and where the dots mean (systematical) zeroes. The squares of the entries of one row sum up to 1 , and the correlations are the sum of the products of the entries along two rows, say for $\small corr(x_1,x_2)=1 \cdot a_1$ or $\small corr(x_2,x_3)=a_1 \cdot b_1 + a_2 \cdot b_2$
If we now want to know the possible range for the correlation $\small corr(x_2,x_3)$ given $\small corr(x_1,x_2)=a$ and $\small corr(x_1,x_3)=b$ then we know immediately that a,b must be the entries in the first column:
$\qquad \small \begin{array} {r|lll} x_1: & 1 & . & . & \\ x_2: & a & a_2 & . \\ x_3: & b & b_2 & b_3 \\ \end{array}$
and by the rule of sum-of-squares = 1 we get
$\qquad \small \begin{array} {r|lll} x_1^*: & 1 & . & . & \\ x_2^*: & a^2 & 1-a^2 & . \\ x_3^*: & b^2 & b_2^2 & 1-b^2-b_2^2 \\ \end{array}$
Here all except the entry $\small b_2$ are fixed or determined by the choice of $\small b_2$, which is also limited to the obvious interval $\small 0 \le b_2^2 \le 1-b^2$.

Let's for simpliness assume a and b are positive values. Then it is also obvious, that we get the possible range for the correlation $\small corr(x_2,x_3)$ if we set $\small x_2$

• to its maximum, that is $\small b_2^2 = 1-b^2, b_2=\sqrt{1-b^2} b_3=0$ $\qquad \small \begin{array} {r|lll} x_1: & 1 & . & . & \\ x_2: & a & \sqrt{1-a^2} & . \\ x_3: & b & \sqrt{1-b^2} & 0 \\ \end{array}$
and $\small corr(x_2,x_3)=a \cdot b + \sqrt{1-a^2}\cdot \sqrt{1-b^2}$
If a=b we have then $\small corr(x_2,x_3)=a^2 + (1-a^2) = 1$

• to some mean value, (which, when we allow only positive values for all entries
is also its minimum) that is $\small b_2^2 = 0, b_3^2=1-b^2,b_3=\sqrt{1-b^2}$ and
$\qquad \small \begin{array} {r|lll} x_1: & 1 & . & . & \\ x_2: & a & \sqrt{1-a^2} & . \\ x_3: & b & 0 & \sqrt{1-b^2} \\ \end{array}$
and $\small corr(x_2,x_3)=a \cdot b + 0$
If a=b we have then $\small corr(x_2,x_3)=a^2 + 0$

• to its minimum (possibly negative, and then not minimal in its absolute value), that is $\small b_2^2 = 1-b^2, b_2=-\sqrt{1-b^2} ,\qquad b_3=0$
$\qquad \small \begin{array} {r|lll} x_1: & 1 & . & . & \\ x_2: & a & +\sqrt{1-a^2} & . \\ x_3: & b & - \sqrt{1-b^2} & 0 \\ \end{array}$
and $\small corr(x_2,x_3)=a \cdot b - \sqrt{1-a^2}\cdot \sqrt{1-b^2} < a\cdot b$

If a=b then we get $\small corr(x_2,x_3)=a \cdot a - \sqrt{1-a^2}\cdot \sqrt{1-a^2} = 2a^2-1 < a^2$ which might also come out to be zero or even negative.

Completely similarly this can be done if more variables in the correlation-matrix are existent, because only the number of rows/columns in the cholesky-factor increases accordingly.

(Remark: for simpliness of the exposition of the principle of that calculations I did not attempt a more exact case-distinction)

• I'm stunned. thanks for this in-depth explanation. It really was the solution I was looking for. Even though you solved it for a different pair of variables, I'm sure I can now find it out myself. Was mainly looking for corr(x1,x3) when all other correlations are known. Anyway, big thanks again! – pokey909 Feb 6 '12 at 16:50
• It is quite easy to find examples where the minimum correlation is $corr(X_1,X_3)=2|a|-1 < a^2$. For example, for $a$ non-negative let $X_1=X_2=X_3=1$ with probability $a/2$, $X_1=X_2=X_3=-1$ with probability $a/2$, $X_1=X_2=1$ and $X_3=-1$ with probability $(1-a)/4$, $X_1=X_2=-1$ and $X_3=1$ with probability $(1-a)/4$, $X_3=X_2=1$ and $X_1=-1$ with probability $(1-a)/4$, $X_3=X_2=-1$ and $X_1=1$ with probability $(1-a)/4$. – Henry Feb 7 '12 at 1:37
• True, thanks for the hint, I was a bit sloppy. This can be seen in the maximum-formula, if we use $\small -\sqrt{1-b^2}$ instead of the positive squareroot, we have then $\small corr(x_2,x_3)=a \cdot b - \sqrt{1-a^2} \cdot \sqrt{1-b^2}$ and for a=b we have then $\small corr(x_2,x_3)=a \cdot a - \sqrt{1-a^2} \cdot \sqrt{1-a^2} = a^2 - (1-a^2) = 2a^2 - 1 < a^2$ I'll append this to the answer. – Gottfried Helms Feb 7 '12 at 2:12

For the first part of your question you were pointed to a quant answer which would give you a lower bound of $a^2-\sqrt{1-a^2}$. The upper bound is clearly $1$, for example if $X_1 = X_3$.

You can apply the same result successively, though note than once $0$ appears in the range of possibilities then you need to take slightly more care on the next step.

Danny Kaplan's textbook on statistics makes something of an issue of this. Things like this: If there's a 95% correlation between $X$ and $Y$ and a 90% correlation between $Y$ and $Z$, then what are the biggest and smallest possible correlations between $X$ and $Z$? The answer comes from the fact that the correlation is the cosine of an angle between two vectors in an inner product space. A 95% correlation means the angle is $\arccos 0.95 \approx 18.19487^\circ$. And 90% means the angle is $\arccos 0.9 \approx 25.8419^\circ$. Add the angles, getting $44.03677^\circ$, and the cosine is $0.7188938$. That's the smallest possible correlation between $X$ and $Z$. Subtract the angles, getting $7.64703^\circ$. The cosine is about $0.9911066$. That's the largest possible correlation between $X$ and $Z$.

• That sounds like the most elegant solution so far. Could you maybe give me a pointer to which book you exactly mean? is it this one macalester.edu/~kaplan/ism ? – pokey909 Feb 6 '12 at 16:52
• I must admit, it sounds coutnerintuitive, since if I have a chain of known correlations like (x1,x2),(x2,x3),...,(xn-1,xn) and want to know the lower bound of (x1,xn) I would have to add all the angles. My intuition tell me that the correlation should steadily decrease but when adding the angles and computing the cosing will at some point wrap below 0 and give me higher negative correlations. My intuition might be wrong though – pokey909 Feb 6 '12 at 16:59
• It's the one at Macalester. – Michael Hardy Feb 6 '12 at 18:02
• If I'm correctly understanding what you mean, your intuition is mistaken. – Michael Hardy Feb 6 '12 at 18:03