# Question about the independence definition.

Why does the independence definition requires that every subfamily of events $A_1,A_2,\ldots,A_n$ satisfies $P(A_{i1}\cap \cdots \cap A_{ik})=\prod_j P(A_{ij})$ where $i_1 < i_2 < \cdots < i_n$ and $j < n$.

My doubt arose from this: Suppose $A_1,A_2$ and $A_3$ such as $P(A_1\cap A_2\cap A_3)=P(A_1)P(A_2)P(A_3)$.

Then

$$P(A_1\cap A_2)=P(A_1\cap A_2 \cap A_3) + P(A_1\cap A_2 \cap A_3^c)$$ $$=P(A_1)P(A_2)(P(A_3)+P(A_3^c))=P(A_1)P(A_2).$$

So it seems to me that if $P(A_1\cap A_2\cap A_3)=P(A_1)P(A_2)P(A_3)$ then $P(A_i\cap A_j)=P(A_i)P(A_j)$, i.e., the biggest collection independence implies the smaller ones. Why am I wrong? The calculations seems right to me, maybe my conclusion from it are wrong?

-
But why is $P(A_1 \cap A_2 \cap A_3^c)=P(A_1) P(A_2)P(A_3^c)$ – user21436 Apr 18 '12 at 22:44
What's the probability that a question will receive three responses within 30 seconds of each other half an hour after the question is posted? – David Mitra Apr 18 '12 at 22:54
All three answers posted so far assume $P(A_3)$ is $0$ or $1$. But one of them asserts that there are less trivial examples. Are there? – Michael Hardy Apr 18 '12 at 23:01
@MichaelHardy Yes, I added one to my answer. – David Mitra Apr 18 '12 at 23:33
@MichaelHardy Yes, there are nontrivial examples. I just posted one. – Dilip Sarwate Apr 18 '12 at 23:53

$P(ABC)=P(A)P(B)P(C)$ does not imply that $P(ABC^C)=P(A)P(B)P(C^C)$, which it seems you're using. Consider, for instance, $C=\emptyset$.

However, see this question.

Another example:

Let $S=\{a,b,c,d,e,f\}$ with $P(a)=P(b)={1\over8}$, and $P(c)=P(d)=P(e)=P(f)={3\over16}$.

Let $A=\{a,d,e\}$, $B=\{a,c,e\}$, and $C=\{a,c,d\}$.

Then

$\ \ \ \ \ \ P(ABC)=P(\{a\})={1\over8}$

and

$\ \ \ \ \ \ P(A)P(B)P(C)= {1\over2}\cdot{1\over2}\cdot{1\over2}={1\over8}$.

But

$\ \ \ \ \ \ P(ABC^C)=P(\{e\})= {3\over16}$

while

$\ \ \ \ \ \ P(A)P(B)P(C^C) = {1\over2}\cdot{1\over2}\cdot{1\over2}={1\over8}$.

In fact no two of the events $A$, $B$, and $C$ are independent.

-
Ok. Look at this. If $P(A\cap B)=P(A)P(B)$ then $P(A\cap B^c)=P(A)-P(A\cap B)=P(A)-P(A)P(B)=P(A)(1-P(B))=P(A)P(B^c)$. So if $C=\phi$ then $ABC=\phi$ and $ABC^c=AB$ so it is still valid. – Rodolfo Apr 18 '12 at 22:54
@Rodolfo: The counterexample offered by David (and Andre and myself) gives an example where the independence formula holds for all three variables but fails for just two of them. In your comment you assume $P(A\cap B)=P(A)P(B)$, which is exactly what you were trying to prove in the first place. – Brett Frankel Apr 18 '12 at 23:01
OK I'm convinced....thank you!!! – Rodolfo Apr 18 '12 at 23:04

When a tetrahedral die is rolled, the outcome is the face on the bottom of the die when it comes to rest. Suppose the four faces are marked $2,3,5,30$, and these numerical outcomes occur with probabilities $\frac{11}{24}, \frac{7}{24}, \frac{5}{24}$ and $\frac{1}{24}$ respectively.

Let $A$, $B$, and $C$ denote the events that the outcome is a multiple of $2$, $3$, and $5$ respectively. Then,

\begin{align*} P(A) &= P\{2,30\} = \frac{1}{2}\\ P(B) &= P\{3,30\} = \frac{1}{3}\\ P(C) &= P\{5,30\} = \frac{1}{4}\\ P(ABC) &= P\{30\} = \frac{1}{24} = P(A)P(B)P(C) \end{align*} but $AB = AC = BC = ABC$ and thus $$P(AB) \neq P(A)P(B), \quad P(AC) \neq P(A)P(C), \quad P(BC) \neq P(B)P(C)$$

On the other hand, if it is a fair die, then $P(A)=P(B)=P(C) = \frac{1}{2}$ and since $P(ABC) = \frac{1}{4}$, we have that

$$P(AB) = P(A)P(B), \quad P(AC) = P(A)P(C), \quad P(BC) = P(B)P(C)$$

but $$P(ABC) \neq P(A)P(B)P(C).$$

-

Here is a quick offhand example. Toss a fair coin once. Let $A$ be the event "head," $B$ the event "tail" and $C$ the event "head and tail." (Of course $C$ has probability $0$.) That counterexample is trivial and boring, so we produce a non-trivial one.

Let $A$ and $B$ be events such that $P(A)=P(B)=1/2$ and $P(A\cap B)=1/5$. It is clear that $A$ and $B$ are not independent and that $P(A\cup B)=4/5$. Let $C=A\cup B$. Then $$P(A\cap B\cap C)=P(A\cap B)=\frac{1}{5}\qquad\text{and}\qquad P(A)P(B)P(C)=\frac{1}{5}.$$

-

Your second equality seems fishy. What if $P(A_3)=0$ and $A_1=A_2$, with $P(A_1)=P(A_2)=P(A_1\cap A_2)=\frac{1}{2}$. This certainly is an example of why you need the full definition.

-
Indeed. That is a problem. So what if I suppose that $A$, $B$ and $C$ differents? What you think? it is valid? – Rodolfo Apr 18 '12 at 23:00
This was an extreme counterexample. But you get essentially the same counterexample any time when $A$ and $B$ are dependent and $P(C)=0$. For example, Andre takes $A$ and $B$ to be complementary events in his post. You can cook up more general examples where $P(C)\neq 0$, but it requires some fussing around with the numbers. – Brett Frankel Apr 18 '12 at 23:04
Yeah...very true. I'm very convinced now. Thank you! – Rodolfo Apr 18 '12 at 23:21