I have $N$ Bernoulli random variables $X_1, ..., X_{N}$ with known parameters $p_1, ..., p_{N}$. I want generate a joint distribution in which these random variables are not independent as I know that joint distribution would just be the product of their marginals.

How can I create this joint distribution that can be updated as I add more Bernoulli random variables?

Dependent Bernoulli trials

^ This seems to be my exact question but I am looking for a much simpler way of defining the distribution than selecting $2^n - 1$ parameters

  • $\begingroup$ Your title says dependent while your questions says not dependent $\endgroup$ – Henry Jul 2 '16 at 23:46
  • $\begingroup$ Editing in 'not $in$dependent' doesn't clarify much; if the joint distribution is a product of marginals, doesn't that make them independent? Hypergeometric trials can be S vs F, or B vs W, if you have blue and white balls in an urn sampled without replacement. But updating would be a problem. For updating, you could have a Markov chain with state space $\{0, 1\}$ in which each value depends on the one just before? Might help if you said what kind of dependence you want, and why. $\endgroup$ – BruceET Jul 3 '16 at 0:43
  • 1
    $\begingroup$ @BruceET In the original model, independence of N Bernoulli random variables was assumed. My goal is to generate a joint distribution without independence and see how things change. I haven't thought about what kind of dependence I want yet. $\endgroup$ – user265634 Jul 3 '16 at 1:14

Based on your most recent comment, I think you should consider a 2-state Markov chain to produce a sequence of random variables $X_i,$ taking values in $\{0, 1\},$ roughly as follows:

Start with a deterministic or random $X_1.$ Then

(i) $P\{X_{i+1} = 1|X_i = 0\} = \alpha,$ and (ii) $P\{X_{i+1} = 0|X_i = 1\} = \beta.$

The parameters $\alpha$ and $\beta$ are the respective probabilities of 'changing state' from one $X_i$ to the next. To avoid certain kinds of deterministic sequences, you may want to use $0 < \alpha, \beta < 1.$ If $\alpha = 1 - \beta,$ then the sequence is independent.

By induction, one can show that $$P\{X_{1+r} = 0|X_1 = 0\} = \frac{\beta}{\alpha+\beta} + \frac{\alpha(1-\alpha - \beta)^r}{\alpha+\beta}.$$ If $|1-\alpha - \beta| < 1$, then in the long run $P\{X_n = 0\} \approx \beta/(\alpha+\beta),$ regardless of the value of $X_1.$

Moreover, there are similar formulas for the '$r$-step transitions' from 0 to 1, 1 to 0, and 1 to 1. Of course, I am skipping over a lot of detail here.

Perhaps there is a rich enough variety of models here to satisfy your curiosity as to what happens when independence fails in this way.

Later chapters in many probability books have a complete development of the theory of 2-state Markov chains. Also there are several good elemeentary books just on Markov chains. [Google '2-state Markov Chain'. One reference among many is Chapter 6 of Suess and Trumbo (2010), Springer, in which I have a personal interest.]


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.