# Correlation between variables

I asked this question on stats SE but did not find a suitable answer so far. Maybe someone can help.

Given n random variables x1,...,xn (one-dimensional). The following is known (corr() = Pearson correlation):

corr(x1,x2) = a
corr(x2,x3) = a


The actual values of the random variables and their covariances are unkown though. Only some of their correlations are known.

From this, is it possible to calculate

corr(x3,x1) = ?


or give an estimate of the lowest possible correlation coefficient

corr(x3,x1) > a


More generally:

Given set of correlations

corr(x_i, x_i+1) with i=[1..c], c<n


is it possible to either directly calculate

corr(x_1, x_c+1)


or give a lower bound a of the coefficient with

corr(x_1, x_c+1) > a

-

I find it most intuitive to use the cholesky-decomposition of some correlation-matrix to look at such questions. The cholesky-decomposition provides a lower triangular matrix which always has (given the variables $\small x_1,x_2,x_3$) the form
$\qquad \small \begin{array} {r|lll} x_1: & 1 & . & . & \\ x_2: & a_1 & a_2 & . \\ x_3: & b_1 & b_2 & b_3 \\ \end{array}$
which can be continued to more rows/columns and where the dots mean (systematical) zeroes. The squares of the entries of one row sum up to 1 , and the correlations are the sum of the products of the entries along two rows, say for $\small corr(x_1,x_2)=1 \cdot a_1$ or $\small corr(x_2,x_3)=a_1 \cdot b_1 + a_2 \cdot b_2$
If we now want to know the possible range for the correlation $\small corr(x_2,x_3)$ given $\small corr(x_1,x_2)=a$ and $\small corr(x_1,x_3)=b$ then we know immediately that a,b must be the entries in the first column:
$\qquad \small \begin{array} {r|lll} x_1: & 1 & . & . & \\ x_2: & a & a_2 & . \\ x_3: & b & b_2 & b_3 \\ \end{array}$
and by the rule of sum-of-squares = 1 we get
$\qquad \small \begin{array} {r|lll} x_1^*: & 1 & . & . & \\ x_2^*: & a^2 & 1-a^2 & . \\ x_3^*: & b^2 & b_2^2 & 1-b^2-b_2^2 \\ \end{array}$
Here all except the entry $\small b_2$ are fixed or determined by the choice of $\small b_2$, which is also limited to the obvious interval $\small 0 \le b_2^2 \le 1-b^2$.

Let's for simpliness assume a and b are positive values. Then it is also obvious, that we get the possible range for the correlation $\small corr(x_2,x_3)$ if we set $\small x_2$

• to its maximum, that is $\small b_2^2 = 1-b^2, b_2=\sqrt{1-b^2} b_3=0$ $\qquad \small \begin{array} {r|lll} x_1: & 1 & . & . & \\ x_2: & a & \sqrt{1-a^2} & . \\ x_3: & b & \sqrt{1-b^2} & 0 \\ \end{array}$
and $\small corr(x_2,x_3)=a \cdot b + \sqrt{1-a^2}\cdot \sqrt{1-b^2}$
If a=b we have then $\small corr(x_2,x_3)=a^2 + (1-a^2) = 1$

• to some mean value, (which, when we allow only positive values for all entries
is also its minimum) that is $\small b_2^2 = 0, b_3^2=1-b^2,b_3=\sqrt{1-b^2}$ and
$\qquad \small \begin{array} {r|lll} x_1: & 1 & . & . & \\ x_2: & a & \sqrt{1-a^2} & . \\ x_3: & b & 0 & \sqrt{1-b^2} \\ \end{array}$
and $\small corr(x_2,x_3)=a \cdot b + 0$
If a=b we have then $\small corr(x_2,x_3)=a^2 + 0$

• to its minimum (possibly negative, and then not minimal in its absolute value), that is $\small b_2^2 = 1-b^2, b_2=-\sqrt{1-b^2} ,\qquad b_3=0$
$\qquad \small \begin{array} {r|lll} x_1: & 1 & . & . & \\ x_2: & a & +\sqrt{1-a^2} & . \\ x_3: & b & - \sqrt{1-b^2} & 0 \\ \end{array}$
and $\small corr(x_2,x_3)=a \cdot b - \sqrt{1-a^2}\cdot \sqrt{1-b^2} < a\cdot b$

If a=b then we get $\small corr(x_2,x_3)=a \cdot a - \sqrt{1-a^2}\cdot \sqrt{1-a^2} = 2a^2-1 < a^2$ which might also come out to be zero or even negative.

Completely similarly this can be done if more variables in the correlation-matrix are existent, because only the number of rows/columns in the cholesky-factor increases accordingly.

(Remark: for simpliness of the exposition of the principle of that calculations I did not attempt a more exact case-distinction)

-
I'm stunned. thanks for this in-depth explanation. It really was the solution I was looking for. Even though you solved it for a different pair of variables, I'm sure I can now find it out myself. Was mainly looking for corr(x1,x3) when all other correlations are known. Anyway, big thanks again! –  pokey909 Feb 6 '12 at 16:50
It is quite easy to find examples where the minimum correlation is $corr(X_1,X_3)=2|a|-1 < a^2$. For example, for $a$ non-negative let $X_1=X_2=X_3=1$ with probability $a/2$, $X_1=X_2=X_3=-1$ with probability $a/2$, $X_1=X_2=1$ and $X_3=-1$ with probability $(1-a)/4$, $X_1=X_2=-1$ and $X_3=1$ with probability $(1-a)/4$, $X_3=X_2=1$ and $X_1=-1$ with probability $(1-a)/4$, $X_3=X_2=-1$ and $X_1=1$ with probability $(1-a)/4$. –  Henry Feb 7 '12 at 1:37
True, thanks for the hint, I was a bit sloppy. This can be seen in the maximum-formula, if we use $\small -\sqrt{1-b^2}$ instead of the positive squareroot, we have then $\small corr(x_2,x_3)=a \cdot b - \sqrt{1-a^2} \cdot \sqrt{1-b^2}$ and for a=b we have then $\small corr(x_2,x_3)=a \cdot a - \sqrt{1-a^2} \cdot \sqrt{1-a^2} = a^2 - (1-a^2) = 2a^2 - 1 < a^2$ I'll append this to the answer. –  Gottfried Helms Feb 7 '12 at 2:12

For the first part of your question you were pointed to a quant answer which would give you a lower bound of $a^2-\sqrt{1-a^2}$. The upper bound is clearly $1$, for example if $X_1 = X_3$.

You can apply the same result successively, though note than once $0$ appears in the range of possibilities then you need to take slightly more care on the next step.

-
Danny Kaplan's textbook on statistics makes something of an issue of this. Things like this: If there's a 95% correlation between $X$ and $Y$ and a 90% correlation between $Y$ and $Z$, then what are the biggest and smallest possible correlations between $X$ and $Z$? The answer comes from the fact that the correlation is the cosine of an angle between two vectors in an inner product space. A 95% correlation means the angle is $\arccos 0.95 \approx 18.19487^\circ$. And 90% means the angle is $\arccos 0.9 \approx 25.8419^\circ$. Add the angles, getting $44.03677^\circ$, and the cosine is $0.7188938$. That's the smallest possible correlation between $X$ and $Z$. Subtract the angles, getting $7.64703^\circ$. The cosine is about $0.9911066$. That's the largest possible correlation between $X$ and $Z$.