44
$\begingroup$

Let's say I want to generate correlated random variables. I understand that I can use Cholesky decomposition of the correlation matrix to obtain the correlated values. If $C$ is the correlation matrix, then we can do the cholesky decomposition:

$LL^{T}=C$

Then I can easily generate correlated random variables:

$LX=Y$,

where $X$ are uncorrelated values and $Y$ are correlated values. If I want two correlated random variables then $L$ is:

$L = \left[ {\begin{array}{*{20}c} 1 & 0 \\ \rho & {\sqrt {1 - \rho ^2 } } \\ \end{array}} \right] $

I understand that this works, but I don't really understand why... My question is: Why does this work?

$\endgroup$
48
$\begingroup$

The co-variance matrix of any random vector $Y$ is given as $\mathbb{E} \left(YY^T \right)$, where $Y$ is a random column vector of size $n \times 1$. Now take a random vector, $X$, consisting of uncorrelated random variables with each random variable, $X_i$, having zero mean and unit variance $1$. Since $X_i$'s are uncorrelated random variables with zero mean and unit variance, we have $\mathbb{E} \left(X_iX_j\right) = \delta_{ij}$. Hence, $$\mathbb{E} \left( X X^T \right) = I$$ To generate a random vector with a given covariance matrix $Q$, look at the Cholesky decomposition of $Q$ i.e. $Q = LL^T$. Note that it is possible to obtain a Cholesky decomposition of $Q$ since by definition the co-variance matrix $Q$ is symmetric and positive definite.

Now look at the random vector $Z = LX$. We have $$\mathbb{E} \left(ZZ^T\right) = \mathbb{E} \left((LX)(LX)^T \right) = \underbrace{\mathbb{E} \left(LX X^T L^T\right) = L \mathbb{E} \left(XX^T \right) L^T}_{\text{ Since expectation is a linear operator}} = LIL^T = LL^T = Q$$ Hence, the random vector $Z$ has the desired co-variance matrix, $Q$.

$\endgroup$
  • $\begingroup$ But how do you specify whether the correlations of those variables are going to be positive or negative? $\endgroup$ – f1r3br4nd Jul 3 '14 at 12:23
  • 4
    $\begingroup$ So it should work for any decomposition of $Q=LL^T$, right? I mean, $L$ does not necessarily have to be triangular, as long as it satisfies that equation. Am I right? $\endgroup$ – Michael Apr 1 '16 at 17:50
  • $\begingroup$ Why do you state that covariance is given by $\mathbb{E} \left(YY^T \right)$? What happen to the $\mathbb{E} \left(Y\right)\mathbb{E} \left(Y\right)^T$ term? $\endgroup$ – Confounded Nov 7 '17 at 11:47
  • $\begingroup$ @Confounded He probably meant centered variables (zero mean) $\endgroup$ – Michael Nov 13 '18 at 20:42
-1
$\begingroup$

The specified correlation, C, can have negative element, i.e. Negative correlation. But, C, must Be positive definite to be a proper corr. matrix. E.g. [ 1 -0.9 0.5 -0.9 1 -0.1 0.5 -0.1 1 ] Is ok, but 0.9,-0.5,0.1 (in all places) produces a negative definite matrix, so not ok. Actually, for all values positive will have the effect that the sign of the second random variable changes.

$\endgroup$

protected by Saad Oct 3 '18 at 14:13

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.