I can't wrap my head around why the Law of Large Numbers applies in the case of coin flips (in this case that the proportion of heads to tails converges to $1:1$ over time). This is because, as I understand it, every permutation of $H$ and $T$ for $n$ flips is supposed to be equally likely, for example, $HTHTHTHT$ is just as likely as $HHHHHHHH$. My main suspicion is that perhaps the outcomes that have a more balanced proportion of heads to tails are more likely (since there are more balanced outcomes than extreme ones in every set of possible outcomes, or at least so I've seen). But even then, I also know that the probability of getting an equal number of heads and tail decreases as the number of coin flips increases, so I end up getting confused. I appreciate any ideas you can share.

  • 3
    $\begingroup$ Every flip sequence is equally likely, but there are many more flip sequences where the number of heads are about 50% than there are sequences where the number of heads are, say, about 10%. $\endgroup$ – Mike Earnest Apr 14 '18 at 0:06
  • 3
    $\begingroup$ Yes, but there's only one string with all heads, and may with say $45-50\%$ heads. Make a list of the possible sequences with, say $6$ flips, and you'll see. $\endgroup$ – saulspatz Apr 14 '18 at 0:07
  • $\begingroup$ The LLN does not at all say you are more and more likely to get an equal number of heads and tails. It merely says the proportion gets close to 0.5. Another view: let $X =$ no. of heads - no. of tails. The LLN does not say the distribution of $X$ narrows as $n$ increases. Instead it says the distribution of $\frac{X}{n}$ narrows as $n$ increases -- and that happens because the denominator is $n$ but the numerator's "width" grows much slower than $n$. Does this help? $\endgroup$ – antkam Apr 14 '18 at 3:33
  • $\begingroup$ A good starting observation is to flip it once. Did it come up heads half the time and tails half the time? No. It came up one side all of the time. $\endgroup$ – samerivertwice Apr 14 '18 at 7:55
  • $\begingroup$ p.s. I've proposed to edit your $0.5$ to $1:1$ as $0.5$ isn't a ratio and it wasn't clear if you meant converges to $0.5:1$ which would of course be untrue, and then it would have made sense that you didn't get it.. $\endgroup$ – samerivertwice Apr 14 '18 at 7:57

Comments by @MikeEarnest and @saulspatz make good points about the increasing number of possibilities as $n$ increases. Perhaps a visualization of what happens with increasing $n$ will help you get an intuitive idea what is going on. There really are lots of possibile sequences of Heads and Tails, but they have to obey the rules of probability and the proportion of Heads must 'converge' toward 1/2.

Let $S_n \sim \mathsf{Binom}(n, 1/2),$ the number of Heads seen in $n$ tosses of a fair coin. The usual estimate of $p = 1/2$ is the proportion of Heads observed $\hat p_n = S_n/n.$

Then a statement in the proof of the Law of Large Numbers (LLN) says that the probability $$Q_n = P\{|\hat p_n - 1/2| < \epsilon\} > 1 - \frac{1}{4n\epsilon^2}.$$ In other words for any small value $\epsilon > 0,$ the error of the estimate $\hat p_n$ of $p = 1/2$ becomes increasingly likely to be smaller than $\epsilon.$

But in the binomial case, we can actually compute the exact probability $Q_n,$ whereas the LLN just gives a bound that suggests the probability of discrepancies smaller than $\epsilon$ approaches $1.$

B = 800;  n = 1:B;  eps = .05;  dlt = .04/B  # dlt to avoid equality in CDF
Q = pbinom(n/2 + n*eps - dlt, n, .5) - pbinom(n/2 -n*eps, n, .5)
plot(n, Q, type="l", xaxs="i");  abline(h=0:1, col="green2")
  curve(1-1/(4*x*eps^2), 1,1000, lwd=2, col="red", add=T)

Below, the jagged black line shows exact probabilities that $|\hat p_n - 1/2| < .05$ are increasing as $n$ increases, while the red curve from the LLN gives a rather loose bound on the probability of such a discrepancy. (The red line goes up to $1$, but off the graph to the right.)

enter image description here

  • $\begingroup$ The actual inequality you stated is not LLN, it is Chebyshev's inequality. $\endgroup$ – Ian Apr 14 '18 at 0:44
  • $\begingroup$ Sure, technically yes. (That or Markov's Inequality.) But the spirit of this intuitive demonstration is the LLN. And a line based on the CLT would go right through the middle of the jagged black curve. $\endgroup$ – BruceET Apr 14 '18 at 1:03

There are two things at work here. It is true that all sequences of $2n$ flips are equally probable, so if you pick a specific sequence that has $n$ heads and $n$ tails it is the same chance to get that as all heads. However, there are lots of sequences that have $n$ heads, so the chance of getting exactly $n$ heads is ${2n \choose n}$ times more than the chance of getting all heads.

As you say, the chance of getting exactly $n$ heads out of $2n$ flips decreases as $n$ increases. The central binomial coefficient, ${2n \choose n} \approx \frac {4^n}{\sqrt {\pi n}}$ so the chance of getting exactly $n$ heads is about $\frac 1{\sqrt{\pi n}}$ which decreases as $n$ increases. What increases as $n$ increases is the chance of being within a fraction $x$ of half. If you want the chance of being within $0.01n$ of even, that increases as $n$ increases. In the normal approximation the fractional standard deviation is proportional to $\frac 1{\sqrt n}$ so more and more of the peak is within any constant fraction of $n$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.