Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I have a random variable X that is a mixture of a binomial and two normals (see what the probability density function would look like (first chart))
and I have another random variable Y of similar shape but with different values for each normally distributed side.

X and Y are also correlated, here's an example of data that could be plausible :

    X     Y
1.  0    -20
2. -5     2
3. -30    6
4.  7    -2
5.  7     2

As you can see, that was simply to represent that my random variables are either a small positive (often) or a large negative (rare) and have a certain covariance.

My problem is : I would like to be able to sample correlated and random values from these two distributions.

I could use Cholesky decomposition for generating correlated normally distributed random variables, but the random variables we are talking here are not normal but rather a mixture of a binomial and two normals.

Many thanks!

share|improve this question
    
is the same bernoulli trial used to generate X and Y - or are they different? and if the latter - are they also correlated in some way? –  ronaf Dec 17 '10 at 4:01
    
@ronaf No it is not the same bernoulli trial. Each variable has a different % to choose between 2 normal distribution (of course for each variable, these % equal to 1 but it might be 80-20 or 70-30, etc.) I would say that each's variable bernoulli trial are not correlated but maybe I am wrong...I am not sure because can't we say that 'everything' is correlated somehow? How could I be sure if they are correlated or not? Kind regards –  ibiza Dec 17 '10 at 13:49
    
Seeing that there is no more answers and comments, I suggest moving this question to stats.stackexchange.com –  mpiktas Dec 20 '10 at 9:18
add comment

2 Answers

If you want to sample $(X,Y)$ you need to find out what is their distribution function. If you know the the conditional densities $p(X|Y)$ and $p(Y|X)$ you can use Gibbs sampling.

share|improve this answer
    
Thanks for your answer, that might well be what I need. I will have a more thorough look tonight, but I am not sure how to calculate p(X|Y) and p(Y|X), maybe you could provide me a small heads-up for processing this meanwhile? Kind regards –  ibiza Dec 17 '10 at 13:42
    
What is the relationship between $X$ and $Y$? Sometimes if you know the relationship the conditional density is very easy to construct. For example if you know that $X$ is normal with mean $Y$ and standard error $\sigma$, then $p(X|Y)~N(Y,\sigma^2)$. –  mpiktas Dec 17 '10 at 14:44
    
It is important to note that X and Y are neither normal...this is what complicates the problem I believe. How can I find the relationship between X and Y if they are not normal? I am able to compute their covariance (and correlation) though, from the sample data that I have on hand at least –  ibiza Dec 17 '10 at 14:53
add comment

Let me say a couple of things which might help here:

(1) The information you give here does not uniquely determine a joint distribution, and in fact information of this sort will never uniquely determine a joint distribution. For instance, if I say that I have two variables X ~ N(0, 1) and Y ~ N(0, 1) with a given correlation between X and Y, it does not follow that (X, Y) follows a multivariate normal distribution -- there are infinitely many joint distributions with this property. (This is in some part a consequence of the fact that covariance only gives us information up to the second order.) You will have to make up a joint density with the desired properties; it will not be uniquely determined.

(2) There are different ways that covariance could be introduced into this model, and they will give the model different theoretical properties. For instance, there could be heavy correlation between the Bernoulli variables that decide which distribution to sample from and no correlation between the actual samples. Conversely the Bernoulli variables for X and Y could be totally uncorrelated, and some or all of the normal/binomial distributions could be correlated to one another. One could also envision a mixture of both scenarios. If you have any more fundamental information about your processes (which I'm assuming you do -- how else would you know that it's a mixture of normal and binomial samples?) then you should use that to decide here.

The main problem here is coming up with a joint PDF -- you are in reality building a mathematical model here, and that's as much art as science. Once you have the joint distribution, there are well-established ways to sample from it. The most straightforward conceptually is to sample x_1, then take a conditional probability distribution for (x_2, x_3, ... x_n), then repeat. There are more sophisticated ways to deal with this problem as well, and in any case most of these algorithms are built into MATLAB or whatever you're going to use to do your sampling already.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.