2
$\begingroup$

I'm trying to learn probability on my own and have recently been studying random variables. The book I'm using provides an explanation of why the criterion for event independence is different than the criterion for random variable independence but I just can't get my head around it.

"Definition 3.8.2 (Independence of many r.v.s). Random variables $X_1 , \ldots , X_n$ are independent if \begin{align} & P (X_1 \leq x_1 , \ldots , X_n \leq x_n ) \\[6pt] = {} & P (X_1 \leq x_1 ) \cdots P (X_n \leq x_n ), \text{ for all } x_1 , \ldots , x_n \in\mathbb R.\end{align} For infinitely many r.v.s, we say that they are independent if every finite subset of the r.v.s is independent. Comparing this to the criteria for independence of $n$ events, it may seem strange that the independence of $X_1 , \ldots , X_n$ requires just one equality, whereas for events we needed to verify pairwise independence for all $\binom{n}{2}$ pairs, three-way independence for all $\binom{n}{3}$ triplets, and so on. However, upon closer examination of the definition, we see that independence of r.v.s requires the equality to hold for all possible $x_1 , \ldots , x_n$ -- infinitely many conditions!"

So somehow, the criteria that each r.v. being tested for independence can take on any value and have the equality still hold allows us to infer that there is tuple-wise independence between each r.v. being tested as well, unlike the criteria for events. Can someone help illuminate this for me?

$\endgroup$
  • 1
    $\begingroup$ For random variables $X,Y,Z$, if we have $P[X\leq x, Y\leq y, Z \leq z] = P[X\leq x]P[Y\leq y]P[Z\leq z]$ for all $z \in \mathbb{R}$, we can take $z\rightarrow\infty$ to obtain $P[X\leq x, Y\leq y] = P[X \leq x]P[Y\leq y]$. On the other hand for events $A,B,C$, if we have $P[A \cap B \cap C] = P[A]P[B]P[C]$, there is no way to take "$c$ to infinity" to obtain $P[A \cap B] = P[A]P[B]$. For example, for any events $A,B,C$ that satisfy $P[C]=0$, we necessarily have $P[A \cap B \cap C] = P[A]P[B]P[C]$ (regardless of whether or not $A, B$ are independent events). $\endgroup$ – Michael Jul 30 at 16:42
  • $\begingroup$ Okay, I do follow that logic... Is this the argument that makes the criteria true, or are there others? The text book makes it sound like I should see that the fact that there are infinite combinations that must hold for the criteria to be met should make it immediately evident to me that tuple-wise independence is a covered case. It just doesn't jump out at me that way. $\endgroup$ – user144841 Jul 30 at 16:53
  • $\begingroup$ The fact that there are infinitely many equations just means that we have more flexibility. It is not enough to say "we have infinitely many equations" to prove things, you must exploit those equations. In this case we are exploiting that by taking $z\rightarrow\infty$ (which would not be possible if the equations only held for finitely many numbers $z$). $\endgroup$ – Michael Jul 30 at 17:00
  • $\begingroup$ Events $A_i$ are independent if and only if the corresponding indicator random variables $1_{A_i}$ are independent, where $1_{A_i} = 1$ when event $A_i$ occurs, $0$ when it does not. $\endgroup$ – Robert Israel Jul 30 at 17:16
1
$\begingroup$

For independent random variables I prefer following definition: $$P(X \in A, Y \in B) = P(X \in A)P(Y \in B)$$ For $\forall A, B$ from given $\sigma$-algebra. Imho, from here is more easy to see, that pairwise independence do not imply joint independence.

| cite | improve this answer | |
$\endgroup$
1
$\begingroup$

This answer summarizes my above comments in a bit more detail.

  • Exercise: Suppose $A, B, C$ are events that satisfy $P[C]=0$. Prove that $$P[A\cap B\cap C] = P[A]P[B]P[C]$$ Thus, the above equation holds whenever $P[C]=0$, even when $A$ and $B$ are not independent events.

  • Let $X,Y,Z$ be random variables that satisfy $$ P[X \leq 4.5, Y\leq 9, Z\leq z] = P[X \leq 4.5]P[Y\leq 9]P[Z\leq z] \quad \forall z \in \mathbb{R}$$ Since there are infinitely many equations here, we can take $z\rightarrow\infty$ to obtain:* $$ P[X \leq 4.5, Y\leq 9] = P[X\leq 4.5]P[Y \leq 9]$$ and hence the events $\{X \leq 4.5\}$ and $\{Y\leq 9\}$ are independent.

  • For events $A,B,C$, if we are told $$ P[A\cap B\cap C] = P[A]P[B]P[C]$$ then we have only one equation, there is no way to "take $c\rightarrow\infty$" to obtain $P[A\cap B] = P[A]P[B]$. (See first exercise for a case when $P[A \cap B] \neq P[A]P[B]$).


*Note: We are using the fact $$ \lim_{z\rightarrow\infty} P[Z\leq z] = 1$$ and more generally, if $\Omega$ is the sample space, we are using the fact $\{Z\leq z\} \nearrow \Omega$ so for any event $A$ we have $A \cap \{Z \leq z\} \nearrow A \cap \Omega$ (and of course $A \cap \Omega = A$) so by the theorem of "continuity of probability": $$ \lim_{z\rightarrow\infty} P[A \cap \{Z \leq z\}] = P[A \cap \Omega] = P[A]$$

| cite | improve this answer | |
$\endgroup$
0
$\begingroup$

The basic definition of independence of random variables $X_1,\ldots,X_n$ says \begin{align} & P (X_1 \in A_1, \ldots , X_n \in A_n ) \\[6pt] = {} & P (X_1 \in A_1 ) \cdots P (X_n \in A_n ), \\ & \text{for all (Borel-)measurable } A_1,\ldots,A_n \subseteq \mathbb R. \end{align} Clearly this implies what was stated in the question. The converse implication takes more work. It's related to the proof that the c.d.f. is enough to determine a probability distribution on (Borel-)measurable subsets of the line.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.