0
$\begingroup$

My book defines mutual independence as: events ${A_1, A_2, ..A_n}$ are mutually independent if for any subset ${A_1, A_2, ..A_m}$ (where $m \leq n$) of these events we have: $$P(A_1 \cap A_2 \cap ...A_m) = P(A_1)P(A_2)...P(A_m)$$ The book also defines discrete random variables $X_1, X_2, ...., X_n$ as mutually independent if $$P(X_1 = r_1 , X_2 = r_2, ...X_n = r_n) = P(X_1= r_1)P(X_2= r_2)...P(X_n= r_n)$$ If $X_1, X_2, ...., X_n$ are continuous with CDFs $F_1(x_1),F_2(x_2),...F_n(x_n)$, then they are mutually independent if: $$F(x_1, x_2..., x_n) = F_1(x_1)F_2(x_2)...F_n(x_n)$$ I'm trying to link these ideas together. Is $X_1 = r_1$ (or $X_1 < x_1$) etc. equivalent to an event in ${A_1, A_2, ..A_n}$ in the first equation? If so, why is there not a requirement to check that all subsets of these events are independent (for example, that $$P(X_1 = r_1 , X_2 = r_2, ...X_m = r_m) = P(X_1= r_1)P(X_2= r_2)...P(X_m= r_m)$$ for any $m \leq n$) per the first definition?

$\endgroup$
1
$\begingroup$

The OP's book's statement that

events ${A_1, A_2, \ldots, A_n}$ are mutually independent if for any subset ${A_1, A_2, \ldots \cap A_m}$ (where $m \leq n$) of these events we have: $$P(A_1 \cap A_2 \cap \ldots \cap A_m) = P(A_1)P(A_2)...P(A_m)$$

is incorrect because the equality stated above must hold for every subset of ${A_1, A_2, ..A_n}$ (e.g. such as $\{A_1, A_3, A_9\}$), not just the subsets $\{A_1, A_2, ..A_m\}$ consisting of the first $m$ of the $n$ events.

A collection of $n$ events $\{A_1, A_2, \ldots, A_{n-1}, A_n\}$ is called a collection of mutually independent events if $$P\big(A_1 \cap A_2 \cap \cdots \cap A_{n-1} \cap A_n\big) = P(A_1)P(A_2)\cdots P(A_{n-1})P(A_n)\tag{*}$$ and furthermore, for every subcollection $\{A_{i_1}, A_{i_2}, \ldots A_{i_k}\}$ of two or more events, it is true that $$P\big(A_{i_1} \cap A_{i_2} \cap \cdots \cap A_{i_k}\big) = P(A_{i_1})P(A_{i_2})\cdots P(A_{i_k})\tag{**} $$

Note that $\{i_1, i_2, \ldots i_k\}$ is a subset of $\{1,2,\ldots, n\}$ and that $$\big(A_1 \cap A_2 \cap \cdots \cap A_n\big) ~~{\subset}~~ \big(A_{i_1} \cap A_{i_2} \cap \cdots \cap A_{i_k}\big)$$ which looks bass ackwards at first glance to many people but is absolutely true: just think about it a bit.

An alternative definition of mutually independent events (which is logically equivalent to the standard one above) is that the following $2^n$ conditions hold: $$P\big(A_1^* \cap A_2^* \cap \cdots \cap A_n^*\big) = P(A_1^*)P(A_2^*)\cdots P(A_n^*)\tag{***}$$ where the $2^n$ equalities that must hold are obtained by replacing each $A_i^*$ with either $A_i$ or by $A_i^c$; your choice for each $i$, but whatever choice you make, you must use the same choice on both sides of $(***)$. With $n$ binary choices, you get $2^n$ equations. One of these $2^n$ equations is of course $(*)$ but note that another one is $$P\big(A_1 \cap A_2 \cap \cdots \cap A_{n-1}\cap A_n^c\big) = P(A_1)P(A_2)\cdots P(A_{n-1})P(A_n^c)$$ which when added to $$P\big(A_1 \cap A_2 \cap \cdots \cap A_{n-1} \cap A_n\big) = P(A_1)P(A_2)\cdots P(A_{n-1})P(A_n)\tag{*}$$ allows us to deduce that $$P\big(A_1 \cap A_2 \cap \cdots \cap A_{n-1}\big) = P(A_1)P(A_2)\cdots P(A_{n-1})$$ which is of the form $(**)$. Similarly, the more general $(**)$ with $k$ events in it is obtained by adding together $2^{n-k}$ of the equations $(***)$.

In summary, the definition of mutually independent events is not just that $(*)$ must hold but that $2^n - n -1$ equalities (one of the form $(*)$ and and the rest of the form $(**)$) must hold. That's how many subsets of $\{1,2,\ldots, n\}$ there are with at least two elements in the subset -- the incredulous are invited to ponder that there are $2^n$ subsets of $\{1,2,\ldots, n\}$ of which $n$ are singletons and one subset is the empty set -- which is an easier way of getting to $2^n - n -1$ than the brute-force adding up of $\binom n2 + \binom n3 + \cdots + \binom{n-1} + \binom nn$. Alternatively, there are $2^n$ conditions $(***)$ that must hold

=================================================

The requirement for independence of $n$ random variables $X_1, X_2, \ldots, X_n$ that $$F_{X_1, X_2, \ldots, X_n}(x_1, x_2,\ldots, x_n) = \prod_{i=1}^n F_{X_i}(x_i)\tag{1}$$ holds for all choices of $x_1, x_2, \ldots, x_n$, that is, at every point in $n$-dimensional space, the $n$-dimensional joint CDF factors into the product of the marginal CDFs. In particular, the equality holds in the limit as some of the $x_i$ increase without bound (and so $F_{X_i}(x_i)$ converges to $(1)$) and thus the left side converges to the marginal CDF of those $x_j$ that retain their finite value while the right side of $(1)$ converges to the product of the $F_{X_j}(x_j)$'s. Since $$\lim_{x_i\to\infty} F_{X_i}(x_i) = 1,$$ we are left with the subset result $$F_{X_{i_1}, X_{i_2}, \ldots, X_{i_m}}(x_{i_1}, x_{i_2},\ldots, x_{i_m}) =\prod_{k=1}^m F_{X_{i_k}}(x_{i_k}),\tag{2}$$ that is, Eq. $(1)$ implies that all the subsets of random variables also adhere to the factorization rule.

Why is the result "different" for random variables than for events where the factorization $$P\big(A_1 \cap A_2 \cap \cdots \cap A_{n-1} \cap A_n\big)=P(A_{i_1})P(A_{i_2})\cdots P(A_{i_k})\tag{*}$$ does not imply the factorization $$P\big(A_{i_1} \cap A_{i_2} \cap \cdots \cap A_{i_k}\big) = P(A_{i_1})P(A_{i_2})\cdots P(A_{i_k})~????\tag{**}$$ Well, the answer is that $(1)$ is not just one instance where the factorization holds but infinitely many instances where we are assuming that the factorization holds. That is, if we have that for one specific (fixed) vector of real numbers $(x_1, x_2, \ldots, x_n)$, it just so happens that the events $A_i = \{X_i = x_i\}$ are independent events and so $F_{X_1, X_2, \ldots, X_n}(x_1, x_2,\ldots, x_n)$ factors into \begin{align}F_{X_1, X_2, \ldots, X_n}(x_1, x_2,\ldots, x_n) &= \prod_{i=1}^n F_{X_i}(x_i)\\ \big\Downarrow~~~~~~~~~~~~~~~~ &~ ~~~~~~~~~~~~\big\Downarrow\\ P\big(A_1 \cap A_2 \cap \cdots \cap A_{n-1} \cap A_n\big) &= P(A_1)P(A_2)\cdots P(A_{n-1})P(A_n) \end{align} that equality does not imply that the events $B_i = \{X_i = x_i^\prime\}$ corresponding to some other vector $(x_1^\prime, x_2^\prime, \ldots, x_n^\prime)$ of real numbers also are independent. In short, $(1)$ involves much stronger assumptions than $(*)$ does; $(1)$ says that infinitely many collections of $n$ events are collections of $n$ mutually independent events while $(*)$ is an assertion about just one of those collections. That is why $(1)$ implies $(2)$ while $(*)$ does not imply $(**)$.

$\endgroup$
  • $\begingroup$ @Yandle I have completed re-written my answer and deleted my previous comment. If you find the revised version satisfactory, you might want to consider deleting your own earlier follow-up questions too. $\endgroup$ – Dilip Sarwate May 21 at 15:19
  • $\begingroup$ From the def. of independence of discrete random variables $P(X_1 = r_1 , X_2 = r_2,...X_n = r_n) = P(X_1= r_1)P(X_2= r_2)...P(X_n= r_n)$, if $X_1,\dots, X_n$ are mutually independent then the equation holds for any combination of $r_i$ in the range of each respective random variable. If $X_i$ can take on $k$ values, we can write $k$ equations $P(X_1 = r_1, ... X_i = r_{i_k},...X_n = r_n) = P(X_1= r_1)...P(X_i = r_{i_k})...P(X_n = r_n)$ and the sum of these equations will just give the marginal PMF of $X_1,\dots,X_{i-1},\dots,X_{i+1} \dots X_n$? (I realize $i$ can equal 1 or n as well). $\endgroup$ – Yandle May 23 at 3:24
  • $\begingroup$ We can keep doing this until we are left with the marginal PMF of the random variables we want factored (i.e. $P(X_i = r_i , X_j = r_j,\dots) = P(X_i= r_i)P(X_j= r_j)\dots$) for any $r_i,r_j\dots$? This is possible because the definition of independence of random variables implies the probability of the intersect of all possible combinations of events $A_i = \{X_i = r_i\}$ can be factored, whereas $P(A_1 \cap A_2 \cap ...A_n) = P(A_1)P(A_2)...P(A_n)$ implies only one specific sequence of say, $A_i = \{X_i = r_i\}$, can be factored, so it doesn't mean a subset of these events can be factored? $\endgroup$ – Yandle May 23 at 3:25
  • $\begingroup$ I agree, but so? You have independent random variables and so the fact that the joint pmf factors everywhere implies that the marginal pdfs also factor. Ditto (with integrals instead of sums) for jointly continuous independent random variables. The distinction between $$P(X_1=r_1,X_2=r_2,\cdots,X_n=r_n)=P(X_1=r_1)P(X_2=r_2)\cdots P(X_n=r_n)$$ for one set of numbers $r_1,r_2,\cdots,r_n$ and $$P(X_1=r_1,X_2=r_2,\cdots,X_n=r_n)=P(X_1=r_1)P(X_2=r_2)\cdots P(X_n=r_n)$$ for ALL sets of numbers $r_1,r_2,\cdots,r_n$ is the crucial point. Independence of RVs is the latter, events the former. $\endgroup$ – Dilip Sarwate May 23 at 3:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.