# Coin Flip Probability Independent or Not?

I give you a hat which has $$10$$ coins inside of it. $$1$$ out of the $$10$$ have two heads on it, and the rest of them are fair. You draw a coin at random from the jar and flip it $$5$$ times. If you flip heads $$5$$ times in a row, what is the probability that you get heads on your next flip?

I tried to approach this question by using Bayes: Let $$R$$ be the event that the coin with both heads is drawn and $$F$$ be the event that $$5$$ heads are flipped in a row. Then \begin{align*} P(R|F) &= \frac{P(F|R)P(R)}{P(F)} \\ &= \frac{1\cdot 1/10}{1\cdot 1/10 + 1/2^5\cdot 9/10} \\ &= 32/41 \end{align*}

Thus the probability that you get heads on the next flip is

\begin{align*} P(H|R)P(R) + P(H|R')P(R') &= 1\cdot 32/41 + 1/2\cdot (1 - 32/41) \\ &= 73/82 \end{align*}

However, according to my friend, this is a trick question because the flip after the first $$5$$ flips is independent of the first $$5$$ flips, and therefore the correct probability is $$1\cdot 1/10+1/2\cdot 9/10 = 11/20$$

Is this true or not?

• +1 to your question, for nice analysis. Independent of that, I agree with your computation of $~\displaystyle \frac{73}{82}.~$ You are right and your friend is wrong. Going forward, with your future MathSE postings, please use MathJax to display your math. Jul 14, 2022 at 3:33
• Your friend is wrong because subsequent flip is dependent on what coin you have, and so you must use the evidence of the first five flips. Jul 14, 2022 at 3:56
• Trick question, the coins are in a hat, not a jar. Jul 14, 2022 at 12:13
• I think there is a very slight ambiguity in the problem statement about what "5 times" refers to. Is it "[You draw a coin at random from the jar and flip it] 5 times" or "You draw a coin at random from the jar and [flip it 5 times]". Your friend would be correct under the first interpretation, and you under the second. (The latter interpretation is probably better, because in the first it should be mentioned that drawing is with replacement, because the whole procedure is possible without replacement and gives yet a third answer.)
– towr
Jul 14, 2022 at 12:24
• I think it's easier to see if you make the scenario more vivid: let's flip 99 times, and on the attempt that all are heads, ask your friend how much they're willing to bet on the 100th is a tail. Jul 15, 2022 at 6:16

To convince your friend that he is wrong, you could modify the question:

A hat contains ten 6-sided dice. Nine dice have scores 1, 2, 3, 4, 5, 6, and the other dice has 6 on every face. Randomly choose one dice, toss it $$1000$$ times, and write down the results. Repeat this procedure many times. Now look at the trials in which the first $$999$$ tosses were all 6's: what proportion of those trials have the $$1000^\text{th}$$ toss also being a 6?

Common sense tells us that the proportion is extremely close to $$1$$. $$\left(\text{The theoretical proportion is}\dfrac{6^{1000}+9}{6^{1000}+54}.\right)$$

But according to your friend's method, the theoretical proportion would be $$\dfrac{1}{10}(1)+\dfrac{9}{10}\left(\dfrac{1}{6}\right)=\dfrac{1}{4}$$. I think your friend would see that this is clearly wrong.

• This is the same approach we use to convince people about the Monte Hall problem. There, we propose 1000 doors; after you pick, the host opens 998 goat-doors. Jul 14, 2022 at 11:38
• The weakness of extreme approaches like this is that once a person conditions a problem incorrectly, the incorrect conditioning can easily stretch to any extreme. I say this from personal experience, having been (briefly) on the wrong side of the argument in my youth. Jul 14, 2022 at 16:52
• Probably best to stick to the coin throws. (You pulled a coin and threw it $1000$ times, every time getting heads: what is the probability that, if you throw it the $1001$st time you will get heads again?) Otherwise, spot on and very instructive IMHO.
– user700480
Jul 14, 2022 at 18:46
• This is a good approach that I like a lot. I would also suggest going the other way -- bring it down to just two coins, one with HH and one with HT. Jul 15, 2022 at 0:38
• @AaronMontgomery : agreed take 2 (not 10) for amount of coins, and, other way down too, take 1 (not 5) for amount of throws ... what do we get in both formulas? Jul 15, 2022 at 1:52

The main idea behind this problem is a topic known as predictive posterior probability.

Let $$P$$ denote the probability of the coin you randomly selected landing on heads.

Then $$P$$ is a random variable supported on $$\{0.5,1\}$$ and has pdf $$\mathbb{P}(P=0.5)=0.9\\ \mathbb{P}(P=1)=0.1$$

Let $$E=\{HHHHH\}$$ denote the "evidence" you witness. The posterior probability that $$P=0.5$$ given this evidence $$E$$ can be evaluated using Bayes' rule:

$$\begin{eqnarray*}\mathbb{P}(P=0.5|E) &=& \frac{\mathbb{P}(E|P=0.5)\mathbb{P}(P=0.5)}{\mathbb{P}(E|P=0.5)\mathbb{P}(P=0.5)+\mathbb{P}(E|P=1)\mathbb{P}(P=1)} \\ &=& \frac{(0.5)^5\times 0.9}{(0.5)^5\times 0.9+1^5\times 0.1} \\ &=& \frac{9}{41}\end{eqnarray*}$$ The posterior pdf of $$P$$ given $$E$$ is supported on $$\{0.5,1\}$$ and has pdf $$\mathbb{P}\left(P=0.5|E\right)=\frac{9}{41} \\ \mathbb{P}\left(P=1|E\right)=\frac{32}{41}$$ Finally, the posterior predictive probability that we flip heads again given this evidence $$E$$ is $$\begin{eqnarray*}\mathbb{P}(\text{Next flip heads}|E) &=& \mathbb{P}(\text{Next flip heads}|E,P=0.5)\mathbb{P}(P=0.5|E)+\mathbb{P}(\text{Next flip heads}|E,P=1)\mathbb{P}(P=1|E) \\ &=& \frac{1}{2}\times \frac{9}{41}+1\times \frac{32}{41} \\ &=& \frac{73}{82}\end{eqnarray*}$$

Let $$R$$ be the event that the coin with both heads is drawn and $$F$$ be the event that 5 heads are flipped in a row.

Further let $$E$$ be the event that the subsequent flip is a head. Then applying the definition of conditional probability, law of total probability, and Bayes' Rule:

\begin{align}\mathsf P(R\mid F) &=\dfrac{\mathsf P(F\mid R)\,\mathsf P(R)}{\mathsf P(F)}\\&=\dfrac{\mathsf P(F\mid R)\,\mathsf P(R)}{\mathsf P(F\mid R)\,\mathsf P(R)+\mathsf P(F\mid R^\complement)\,\mathsf P(R^\complement)}\\[2ex]\mathsf P(E\mid F) &= \dfrac{\mathsf P(E, F)}{\mathsf P(F)}\\&=\dfrac{\mathsf P(E\mid R)\,\mathsf P(F\mid R)\,\mathsf P(R)+\mathsf P(E\mid R^\complement)\,\mathsf P(F\mid R^\complement)\,\mathsf P(R^\complement)}{\mathsf P(F)}\\&=\mathsf P(E\mid R)\,\mathsf P(R\mid F)+\mathsf P(E\mid R^\complement)\,\mathsf P(R^\complement\mid F)\end{align}

Which is what you used.

However, according to my friend, this is a trick question because the flip after the first 5 flips is independent of the first 5 flips,

The six flips are conditionally independent for a given coin.   You do not know what coin you have, but you do have the evidence of the first five flips; which you cannot ignore the way your friend suggested.

Personally, I would have gone straight to:

\begin{align}\mathsf P(E\mid F) &= \dfrac{\mathsf P(E, F)}{\mathsf P(F)}\\&=\dfrac{\mathsf P(E,F\mid R)\,\mathsf P(R)+\mathsf P(E,F\mid R^\complement)\,\mathsf P(R^\complement)}{\mathsf P(F\mid R)\,\mathsf P(R)+\mathsf P(F\mid R^\complement)\,\mathsf P(R^\complement)}\\&=\dfrac{1^6/10+(1/2)^6(9/10)}{1^5/10+(1/2)^5(9/10)}\\&=\dfrac{73}{82}\end{align}

The evidence of obtaining five heads among the first five flips, suggest a high probability that the next flip will also be a head.

One approach is to modify the problem to something that's more "obvious" in order to break through the misconception about the original problem. Another answer suggested using larger numbers, namely 1000 die rolls rather than 5 coin flips. I will suggest using smaller numbers.

Suppose there are just two coins in the hat: one fair (whose two sides we can call $$H$$ and $$T$$), one double-headed (whose two sides we can call $$H_1$$ and $$H_2$$). You draw a coin and flip it once. It comes up heads. What's the probability it comes up heads again if you flip it again?

You have four equally likely outcomes for drawing the coin and tossing it:

• Draw the fair coin, toss $$T$$.
• Draw the fair coin, toss $$H$$.
• Draw the double-headed coin, toss $$H_1.$$
• Draw the double-headed coin, toss $$H_2.$$

Given that you saw a head on the first toss, we know we're in one of the last three cases. The results of the next toss give us six equally likely outcomes:

• Draw the fair coin, toss $$H$$, then $$T$$.
• Draw the fair coin, toss $$H$$, then $$H$$.
• Draw the double-headed coin, toss $$H_1,$$ then $$H_1.$$
• Draw the double-headed coin, toss $$H_1,$$ then $$H_2.$$
• Draw the double-headed coin, toss $$H_2,$$ then $$H_1.$$
• Draw the double-headed coin, toss $$H_2,$$ then $$H_2.$$

So the odds of seeing heads on the next toss are five to one, as opposed to three to one (which they would be if your friend's way of thinking were correct).

Putting more coins in the hat reduces the odds for the second toss, but not enough to make them the same as the odds on the first toss, because only the fair coins can be eliminated on the first toss. Requiring more heads to be tossed increases the odds.

Here's a very simple counterexample to your friend's logic: Consider a modified scenario in which you got tails at least once, instead of getting all heads. Obviously, a two-headed coin will never come up tails, so you must have one of the fair coins, which means the probability of heads on the next flip is 1/2. But your friend's logic would still give the same answer of 11/20 in this case.

• In this case, I suspect the friend would claim (incorrectly): "If a tail came up within the first 5 tosses, then we must have the normal coin, so the probability that the 6th toss is heads is 1/2. But if the first 5 tosses are all heads, then we could have either coin, so the probability that the 6th toss is heads is 11/20."
– Dan
Jul 15, 2022 at 0:58

The flips for a particular fixed coin are independent, but once you combine more than one coin, they are no longer independent.

Something else that may be confusing your friend is the distinction between dependent in a statistical sense versus in a causal sense. The next coin flip does not depend on the previous ones is a causal sense of previous heads causing future ones, but in Statistics, "dependent" just refers to things going together. Dependence is all about information. If I give you a fair coin, then seeing what the first five flips are doesn't give you any new information regarding what the last flip will be. But if I don't tell you whether it's a double-headed coin or not, then seeing five heads does give you useful information.

Suppose that someone flipped each coin six times, and for each head they took a black marble, and for each tail a white one. Then they took a box for each coin and put the corresponding marbles in that box. So now you come along and see ten boxes. You take one and take out five marbles at random. They're all black. What's the probability that the sixth one is also black? The black marbles are clustered together, so seeing five together tells you that it's likely another one will be there as well.

Another argument: suppose you had seen two heads and three tails. Would you calculate the probability as 11/20? No, because clearly you don't have the double-headed coin, so the probability is now 1/2. The probability went down from 11/20 to 10/20. Where did that extra 1/20 "go"? The total probability over all possibilities of five flips is 11/20, so if one of those possibilities is giving you a lower probability, then there must be another with a higher probability. If the probability goes down when we see tails, then it must go up when we see no tails.

I think a simple example to show your friend must be wrong is to ask them what their answer would be if you had flipped the coin 5 times and gotten 5 'tails'. The answer in that case would be $$\frac{1}{2}$$, not $$\frac{11}{20}$$. They're right that it would not be $$\frac{1}{64}$$ because each toss is independent of the previous toss, but the odds of having the double-headed coin has dropped to zero. This shows that the odds of having the double-headed coin are important.

And the odds of grabbing a double-headed coin are the same $$\frac{1}{10}$$ as grabbing any other coin, but after one toss the odds of having a double-headed coin are different depending on the result:

• TAILS - $$\frac{9}{20}$$ chance, in all 9 cases you have a fair coin
• HEADS - $$\frac{11}{20}$$ chance, in $$\frac{9}{11}$$ cases you have a fair coin, in $$\frac{2}{11}$$ cases you have the double-headed coin.

So to get the odds of that second toss being heads or tails, you have to look at the odds of which coin you have after the first toss:

• $$\frac{9}{20}$$ tails first * $$\frac{9}{9}$$ fair coin * $$\frac{1}{2}$$ tails next
• $$\frac{9}{20}$$ tails first * $$\frac{9}{9}$$ fair coin * $$\frac{1}{2}$$ heads next
• $$\frac{11}{20}$$ heads first * $$\frac{9}{11}$$ fair coin * $$\frac{1}{2}$$ heads next
• $$\frac{11}{20}$$ heads first * $$\frac{2}{11}$$ double-headed coin * $$\frac{1}{1}$$ heads next

Your friend seems to think that the formula is:

$$P(H) = P(1)P(H|1)+P(2)P(H|2)$$

where 1 and 2 are the coins with two heads or a heads and a tails, respectively. In other words, just take the probability of getting heads with each coin and multiply by the fraction of each coin, then sum the two.

Here is an easy demonstration that your friend's reasoning is incorrect.

Suppose that you instead have one coin with two heads and nine coins with two tails. You flip five heads in a row. Your friend's reasoning tells you that the probability that you get heads on your next flip should be:

$$P(H) = 1\times(1/10)+0\times(9/10)=1/10$$

However, this is impossible, since having any heads at all means that you have chosen the one coin with two heads, so the probability must be 1 (100%). As such, your friend's reasoning must be wrong.

Your friend reasoning might be translatd into the fact that randomly choosing a coin from a mix of coins is equivalent to have a coin whose probability is given by the ( weighted) average of the probability of each coin in the mix $$\left\{ \begin{array}{l} p_{eq} = \frac{{\sum\limits_k {n_k p_k } }}{{\sum\limits_k {n_k } }} \\ q_{eq} = \frac{{\sum\limits_k {n_k q_k } }}{{\sum\limits_k {n_k } }} = \frac{{\sum\limits_k {n_k \left( {1 - p_k } \right)} }}{{\sum\limits_k {n_k } }} = 1 - p_{eq} \\ \end{array} \right.$$

That is totally true when we are tossing the extracted / equivalent coin once.

However the "equivalence" breaks away (in general) when we are considering a sequence of more than one tosses

The scheme below shows what happens at the second toss in your example, of the same previously selected or "equivalent" coin

And more, there is not actually an "equivalent" coin that tossed two times might give the correct result, since the system $$\left\{ \begin{array}{l} p^2 = 13/40 \\ q^2 = \left( {1 - p} \right)^2 = 9/40 \\ pq = p\left( {1 - p} \right) = 9/40 \\ \end{array} \right.$$ does not have real solutions.