Convergence of the Law of large number

I have an "honest" coin and I toss it many times. According to the Law of large numbers, the more I toss the more the head occurrences align to the head probability ($0.5$). Is there a way to measure this convergence?

Because I noticed that, as $N=\text{[number of tosses]}$ grows up, number of heads $\approx$ number of tails, but (number of heads $-$ number of tails) becomes larger.

For example, if $N=100$, it is probable to have $60$ heads and $40$ tails; if $N=1500$, $788$ heads and $712$ tails ---> number of heads better aligns to head probability, but (number of heads $-$ number of tails) becomes larger.

• The behavior of $$M_N=\text{[# of heads]}-\frac{N}{2}=\frac{\text{[# of heads]}-\text{[# of tails]}}{2}$$ is a subject of both the central limit theorem (which tells that $M_N$ is asymptotically normal and typically has order of $\sqrt{N}$) and the large deviation theory (which tells that $\mathbf{P}[M_N > Nx] \approx e^{-Nf(x)}$ for a suitable function $f$. – Sangchul Lee May 27 '18 at 10:01
• Hi, do you know where i can find an accurate explanation of what you wrote ? – Qwerto May 28 '18 at 8:24

If $H$ is the number of heads and $T$ the number of tails from a fair coin, with $H+T=N$, then

• $H$ has expectation $\frac{N}2$ and variance $\frac{N}4$ and standard deviation $\frac{\sqrt{N}}{2}$, increasing with $N$
• $H-T$ has expectation $0$ and variance ${N}$ and standard deviation $\sqrt{N}$, increasing with $N$
• $\frac{H-T}{H+T}$ has expectation $0$ and variance $\frac{1}{N}$ and standard deviation $\frac{1}{\sqrt{N}}$, decreasing with $N$
• $\frac{H}{H+T}$ has expectation $\frac{1}2$ and variance $\frac{1}{4N}$ and standard deviation $\frac{1}{2\sqrt{N}}$, decreasing with $N$

The Law of Large Numbers is consistent with the last of these

You have correctly observed the increasing/decreasing contrast depending on whether you concentrate on counts or proportions

• Hi, where can i find an accurate explanation of what you said ? – Qwerto May 28 '18 at 8:26
• @Qwerto - I believe my statements are accurate and are based on basic properties of means and variances - perhaps you were looking for a different word. – Henry May 28 '18 at 17:35

I noticed that, as N=[number of tosses] grows up number of heads ≈ number of tails, but (number of heads − number of tails) becomes larger.

Yes. The law of large numbers tells you that the sample mean converges (in probability!) to the true mean, that is, that $\hat p = H/N \to p$, more in detail, that the probability of having $|p - \hat p|>\epsilon$ tends to zero as $N$ grows.

But this does not imply that the difference between tails and heads $|H - T|=| N - 2H|$ tends to zero. (It only tends to zero when divided by $N$).

Is there a way to measure this convergence?

A quick (but not tight) bound is given by Chebyshev's inequality. In our case, it says that

$$P( |\frac{H}{N} - \frac12| \ge \epsilon) \le \frac{1}{4 N \epsilon^2}$$

or, calling $\Delta = T-H$

$$P\left( \frac{|\Delta|}{N} \ge \epsilon\right) \le \frac{1}{ N \epsilon}$$

More (very) precise estimation of the probabability of that deviation can be obtained by the Central Limit theorem.

• Hi. The convergence i was talking about it the (number of heads - number of tails) convergence – Qwerto May 28 '18 at 8:23
• @Qwerto Well, "(number of heads - number of tails)" is what I called $- \Delta$ And I said it does not tend to zero - only if divided by $N$ – leonbloy May 28 '18 at 12:40