0
$\begingroup$

I am trying to determine the probability that a particular event succession belongs to a given probability distribution PDF.

Simplifying the problem, let's say I have a biased coin which gives:

  • heads 2/3 of the time
  • tails 1/3 of the time

Since heads are more likely, if I had to make a bet for a particular event succession, I would go for getting $N=10$ heads in a row: $$P(\text{N heads in a row}) = \left(\frac{2}{3}\right)^N \approx 1.7\%$$

Nonetheless, intuitively I would expect that given $N=10$ coin tosses, $2N/3$ of these would be heads and the other $N/3$ would be tails:

$$P(\text{2N/3 heads and N/3 tails}) = \frac{N!}{(2N/3)!\ (N/3)!}\ \left(\frac{2}{3}\right)^{2N/3}\ \left(\frac{1}{3}\right)^{N/3} \approx 26\%$$

However, the probability of a particular sequence with 2N/3 heads and N/3 tails is lower: $$P(\text{a particular sequence 2N/3 heads N/3 tails}) = \left(\frac{2}{3}\right)^{2N/3}\ \left(\frac{1}{3}\right)^{N/3} \approx 0.22\%$$

Denoting heads$=1$ and tails$=0$, which of the following event successions would be more likely to be a sample drawn form this particular biased coin (taking the previous considerations into account)?

  • 10 heads in a row: $0000000000$
  • 7 heads and 3 tails: $0010011000$

EDIT:

I would say that $0000000000$ is more likely to belong to the PDF since it is the most probable succession.

However, since a sequence that has $2/3$ heads and $1/3$ tails is the typical (most likely) outcome of the coin, $0010011000$ is a good representative of the PDF of the coin, and thus I feel like to should be more likely to belong to its Probability Distribution than the generic succession $0000000000$.

I would appreciate it a lot if anyone could clarify this point. :)

$\endgroup$

2 Answers 2

3
$\begingroup$

$$ \mathbb P(0000000000) = \left( \frac{2}{3} \right)^{10} \approx 1.734\%$$

$$ \mathbb P(0010011000) = \left( \frac{2}{3} \right)^{7} \cdot \left( \frac{1}{3} \right)^{3} \approx 0.216\%$$

Hence, the first one would be more likely.

Note that the second value is the probability of the exact same case $0010011000$ and does not account for other permutations. You can account for other permutations by $$ \mathbb P = \frac{10!}{7!\cdot3!} \cdot \mathbb P(0010011000) \approx 25.92\% $$

$\endgroup$
2
  • $\begingroup$ Thank you! Yes, this was my guess as well. Nonetheless, wouldn't it be odd to say that getting all heads is the most likely event? (it would be extremely lucky!) Furthermore, this is the probability of each succession occurring, whereas I would be interested in the probability of each succession belonging to the distribution instead (not sure if they are the same thing). $\endgroup$ Jul 13, 2022 at 7:31
  • $\begingroup$ Getting all the heads is the most likely event because the probability of any other particular event is even lower. $\endgroup$
    – axr
    Jul 13, 2022 at 7:36
0
$\begingroup$

Although it seems paradoxical, when comparing to another biased coin

  • 7 heads and 3 tails will always be more likely to belong to our original coin (coin A) than to any other coin (coin B).
  • for most of other coins a sequence of 10 heads is more likely to belong to our original coin than a sequence that contains 7 heads and 3 tails.

That is, for most of other coins the generic succession $10$ heads in a row is more likely to be a sample drawn from our original coin than any individual 7 heads and 3 tails sequence.

For any other coin with probabilities $P_\text{coin B}(\text{head})$ and $P_\text{coin B}(\text{tail})$, the probability of getting a particular sequence of 7 heads and 3 tails is: $$ P_\text{coin B}(\text{7 heads 3 tails})=\left[P_\text{coin B}(\text{head})\right]^7\cdot\left[P_\text{coin B}(\text{tail})\right]^3 $$

This function attains the maximum when $P_\text{coin B}(\text{head}) = 2/3$ and $P_\text{coin B}(\text{tail}) = 1 - P_\text{coin B}(\text{head})=1/3$ so for any other coin getting 7 heads and 3 tails is less likely.

On the other hand, for our second coin the probability of getting $10$ heads in a row is $$ P_\text{coin B}(\text{10 heads})=\left[P_\text{coin B}(\text{head})\right]^{10} $$

Using Bayes' Theorem for two competing hypotheses, the probability that a sequence was generated using coin A is given by $$ P(\text{sequence comes from coin A}) = \frac{P_\text{coin A}(\text{sequence})}{P_\text{coin A}(\text{sequence}) + P_\text{coin B}(\text{sequence})} $$

Using this formula, $$ P(\text{7 heads and 3 tails comes from coin A}) = \frac{0.22\%}{0.22\% + \left[P_\text{coin B}(\text{head})\right]^7\cdot\left[P_\text{coin B}(\text{tail})\right]^3} $$

$$ P(\text{10 heads comes from coin A}) = \frac{1.7\%}{1.7\% + \left[P_\text{coin B}(\text{head})\right]^{10}} $$

For $P_\text{coin B}(\text{head}) = \frac{2}{3} \Rightarrow$ both coins A and B have the exact same probabilities for tails and heads so $P(\text{7 heads 3 tails comes from coin A}) = \frac{1}{2}$ and $P(\text{10 heads comes from coin A}) = \frac{1}{2}$.

For $P_\text{coin B}(\text{head}) < \frac{2}{3} \Rightarrow$ we have that $P(\text{7 heads 3 tails comes from coin A})$ will be smaller than $P(\text{10 heads comes from coin A})$.

  • This happens because for small $P_\text{coin B}(\text{head})$ the probability of tossing 10 heads in a row decreases dramatically $\sim x^{10}$ whereas the probability of getting a sequence of 7 heads and 3 tails does not drop towards zero as sharply $\sim x^7$. So if we get 10 heads in a row, it most likely comes from coin A, but we cannot be so sure about getting 7 heads and 3 tails.

For $P_\text{coin B}(\text{head}) > \frac{2}{3} \Rightarrow$ we have that $P(\text{7 heads 3 tails comes from coin A})$ will be larger than $P(\text{10 heads comes from coin A})$.

  • This happens because for larger $P_\text{coin B}(\text{head})$ the probability of tossing $10$ heads in a row sky-rockets towards $1$ as $\sim x^{10}$, whereas getting 7 heads and 3 tails becomes less and less likely for coin B. Thus, in this region, most of the 10 head sequences will belong to coin B and most of the 7 heads and 3 tails to coin A.

TL;DR In short, for biased coins that have $P_\text{coin B}(\text{head}) < \frac{2}{3}$ (which is most of other coins) the probability of getting 10 heads in a row plummets towards zero, whereas the probability of a sequence of 7 heads and 3 tails does not decrease as fast. Consequently, any sequence with 10 heads in a row is more likely to belong to our original coin which had $P_\text{coin A}(\text{head}) = \frac{2}{3}$, but we cannot be so sure about a sequence of 7 heads and 3 tails.


(Take the numbers with a grain of salt)

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .