# Generating random values from non-normal and correlated distributions

I have a random variable X that is a mixture of a binomial and two normals (see what the probability density function would look like (first chart))
and I have another random variable Y of similar shape but with different values for each normally distributed side.

X and Y are also correlated, here's an example of data that could be plausible :

    X     Y
1.  0    -20
2. -5     2
3. -30    6
4.  7    -2
5.  7     2


As you can see, that was simply to represent that my random variables are either a small positive (often) or a large negative (rare) and have a certain covariance.

My problem is : I would like to be able to sample correlated and random values from these two distributions.

I could use Cholesky decomposition for generating correlated normally distributed random variables, but the random variables we are talking here are not normal but rather a mixture of a binomial and two normals.

Many thanks!

• is the same bernoulli trial used to generate X and Y - or are they different? and if the latter - are they also correlated in some way? – ronaf Dec 17 '10 at 4:01
• @ronaf No it is not the same bernoulli trial. Each variable has a different % to choose between 2 normal distribution (of course for each variable, these % equal to 1 but it might be 80-20 or 70-30, etc.) I would say that each's variable bernoulli trial are not correlated but maybe I am wrong...I am not sure because can't we say that 'everything' is correlated somehow? How could I be sure if they are correlated or not? Kind regards – ibiza Dec 17 '10 at 13:49
• Seeing that there is no more answers and comments, I suggest moving this question to stats.stackexchange.com – mpiktas Dec 20 '10 at 9:18

If you want to sample $(X,Y)$ you need to find out what is their distribution function. If you know the the conditional densities $p(X|Y)$ and $p(Y|X)$ you can use Gibbs sampling.

• Thanks for your answer, that might well be what I need. I will have a more thorough look tonight, but I am not sure how to calculate p(X|Y) and p(Y|X), maybe you could provide me a small heads-up for processing this meanwhile? Kind regards – ibiza Dec 17 '10 at 13:42
• What is the relationship between $X$ and $Y$? Sometimes if you know the relationship the conditional density is very easy to construct. For example if you know that $X$ is normal with mean $Y$ and standard error $\sigma$, then $p(X|Y)~N(Y,\sigma^2)$. – mpiktas Dec 17 '10 at 14:44
• It is important to note that X and Y are neither normal...this is what complicates the problem I believe. How can I find the relationship between X and Y if they are not normal? I am able to compute their covariance (and correlation) though, from the sample data that I have on hand at least – ibiza Dec 17 '10 at 14:53

Let me say a couple of things which might help here:

(1) The information you give here does not uniquely determine a joint distribution, and in fact information of this sort will never uniquely determine a joint distribution. For instance, if I say that I have two variables X ~ N(0, 1) and Y ~ N(0, 1) with a given correlation between X and Y, it does not follow that (X, Y) follows a multivariate normal distribution -- there are infinitely many joint distributions with this property. (This is in some part a consequence of the fact that covariance only gives us information up to the second order.) You will have to make up a joint density with the desired properties; it will not be uniquely determined.

(2) There are different ways that covariance could be introduced into this model, and they will give the model different theoretical properties. For instance, there could be heavy correlation between the Bernoulli variables that decide which distribution to sample from and no correlation between the actual samples. Conversely the Bernoulli variables for X and Y could be totally uncorrelated, and some or all of the normal/binomial distributions could be correlated to one another. One could also envision a mixture of both scenarios. If you have any more fundamental information about your processes (which I'm assuming you do -- how else would you know that it's a mixture of normal and binomial samples?) then you should use that to decide here.

The main problem here is coming up with a joint PDF -- you are in reality building a mathematical model here, and that's as much art as science. Once you have the joint distribution, there are well-established ways to sample from it. The most straightforward conceptually is to sample x_1, then take a conditional probability distribution for (x_2, x_3, ... x_n), then repeat. There are more sophisticated ways to deal with this problem as well, and in any case most of these algorithms are built into MATLAB or whatever you're going to use to do your sampling already.

I like your question. Please check the 'rejection method' in the link below. It is very simple, intuitive and precise.

http://web.mit.edu/urban_or_book/www/book/chapter7/7.1.3.html

We can generate samples from any distribution (not necessarily standard) using this method, given that they obey two conditions:

1. Assumes values only within a finite range.
2. Has a PDF that is bounded (i.e., does not go to infinity for any value of the random variable).

After you generate samples for both X and Y, you can use gaussian coupula to induce correlation, i.e.,

1. Generate random normal variables, both with mean 0, variance 1.
2. Induce correlation using cholesky.
3. Transform them to uniform distribution using standard normal CDF.
4. Use the empirical CDF (or a piece-wise defined CDF, if you have it) from your data to invert the uniform variables to get the final correlated sample.

I think t-copula works well as you have heavy tails. Try others for different dependence structures.

I think your graphs and data are apt to apply this. I know this question is very old, but I hope this helps others.

• Hi.. I have a related problem with data generation. I have a small sample in the form of two columns of a relation (they are correlated, lets say salary and tax). I need to generate more samples that mimic the same correlation as the small sample. Could I follow the approach you have outlined? Also, could you please mention some good references to the approach? Thank you. – user2761431 Sep 12 '14 at 19:00
• hi, i was inactive for long time. yes, you can use it. but in the last step you can use the empirical cdf as i assume you are not going to fit your data to any standard distribution. this is essentially a way of generating a gaussian copula. Hope you tried it and it worked! – Raghava Mar 11 '18 at 22:47