6
$\begingroup$

Consider a sample space of two coin tosses = $\{HH, HT, TH, TT\}$.
Suppose that the coin is fair and therefore every outcome has probability $\frac{1}{4}$.

Now, consider another probability model where the coin is biased and head occurs with probability $\frac{3}{4}$.
In this model the corresponding probabilities = $\{\frac{9}{16}, \frac{3}{16}, \frac{3}{16}, \frac{1}{16}\}$.

Definition of independence $P(A \cup B) = P(A)P(B)$
Event $A$ = first coin toss results in head = $\{HT, HH\}$
Event $B$ = both coin toss results in same outcome = $\{HH, TT\}$

First model:
$P(A) = P(\{HT,HH\}) = \frac{1}{4} + \frac{1}{4} = \frac{1}{2}$
$P(B) = P(\{HH,TT\}) = \frac{1}{4} + \frac{1}{4} = \frac{1}{2}$
$P(A \cup B) = P(\{HH\}) = \frac{1}{4} = \frac{1}{2}\times\frac{1}{2} = P(A)P(B)$
Therefore, in this probability model events $A$ and $B$ are independent.

Second model :
$P(A) = P(\{HT,HH\}) = \frac{3}{16} + \frac{9}{16} = \frac{12}{16}$
$P(B) = P(\{HH,TT\}) = \frac{9}{16} + \frac{1}{16} = \frac{10}{16}$
$P(A \cup B) = P(\{HH\}) = \frac{9}{16}$ which is not equal to $P(A)P(B) = \frac{15}{32}$
Therefore, in this probability model events $A$ and $B$ are not independent

Thus, independence of the events depend upon the underlying probability model.
However, I am not getting the intuition. Can anyone explain?

Also Bayes' rule and law of total probability is applicable irrespective of underlying probability model so why the independence of events differs from one probability model to another?

$\endgroup$
  • 3
    $\begingroup$ In the second model, getting a head on the first substantially increases the chance of a match. To put it another way, getting a tail on the first toss makes a match unlikely. This is clearer if the probability of a head is say $99/100$. $\endgroup$ – André Nicolas Jan 9 '15 at 8:18
  • 2
    $\begingroup$ It is better for the intuition to think in terms of $\Pr(B|A)\ne \Pr(B)$ than in terms of $\Pr(A\cap B)\ne \Pr(A)\Pr(B))$. $\endgroup$ – André Nicolas Jan 9 '15 at 8:24
3
+100
$\begingroup$

The individual coin tosses are still independent, but now the outcome of the first toss is informative of your odds on the second. Imagine a bet based on a slightly modified version of your experiment: You toss a coin, see the results, then determine how much you want to bet that the second toss matches the first toss.

Under Model 1, the amount you would bet wouldn't change based on the outcome of the first toss, since it is equally likely that you could get a macth or not. This is not so under Model 2, where you would be willing to bet less money if the first toss were Tails. So, under Model 2, the first toss tells you something.

You seem to have the mathematical details down, so I'm not sure if you need some sort of alternative derivation. Independence depends on the underlying model, as you indicated. In particular, if the conditional distribution is simply proportional to the unconditional distribution, then you have independence.

$\endgroup$
  • $\begingroup$ But if I know the probabilities, which is the case here, aren't the coin tosses independent? I would always bet for heads since its probability is more than the tails. This doesn't have to depend upon the previous outcomes if I know the probability. $\endgroup$ – Curious Jan 13 '15 at 4:00
  • $\begingroup$ @Curious if the bet were that the second toss were heads, I'd agree with you. But the bet is that the second toss matches the first, which depends critically on what the first toss in (in Model 2). Even though a sequence of random variables is iid does not imply that every event constructed from such a sequence is independent. $\endgroup$ – user76844 Jan 13 '15 at 4:12
  • $\begingroup$ In Model 2, probability of heads is 3/4. Now consider conditional probability P(B/A) = P(A and B)/P(A) = (9/16)/(12/16) = 3/4. Given that first coin toss result in Heads, the probability of second coin toss resulting in heads is 3/4. So clearly, the outcome of first coin toss haven't affected the probability of heads which is still 3/4. $\endgroup$ – Curious Jan 13 '15 at 5:43
  • $\begingroup$ @Curious What I am trying to point out is that you will revise your bet upon knowing the first toss. Before any tosses, your probability of getting a matching pair is $P(HH)+P(TT)=10/16$...pretty good. Now, if you throw a tail on the first throw, $P(TT|T_1)=1/4<10/16$, so you would not be willing to wager as much on the bet "you will end up getting matching pair". Note that this is not what happens under Model 1, where you end up with a probability of $1/2$ either way. I'm trying to give some intuition to the math you've already laid out...correctly. $\endgroup$ – user76844 Jan 13 '15 at 6:12
  • $\begingroup$ @Curious you may also want to look up "conditional independence": en.wikipedia.org/wiki/Conditional_independence $\endgroup$ – user76844 Jan 13 '15 at 6:16
1
$\begingroup$

Both events have $HH$ in them. Thus, if prob of heads is higher than $0.5$, there is a stronger chance they will both happen. Another way to think of independence is:

Originally,

$P(B \cup A) = P(A)P(B|A)$

$A$ and $B$ are independent iff $P(B|A) = P(B)$

If $A$ occurs, there is a chance because it is due to the $HH$ in it. If $A$ occurs due to the $HH$, then $B$ occurs. In second model, the possibility of heads is higher and thus the possibility of $HH$ is higher.

Possibly related: http://en.wikipedia.org/wiki/Bertrand's_box_paradox

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.