Short version

According to the law of large numbers, how many samples do I need to take to reach the half-life of the convergence towards the mean?

In other words: How big is a large sample?

Long version

I am having a lot of fun getting back to learning math and I just watched a video about the law of large numbers. In basic words it means that the more random samples you take the closer your average value will be to the median (or expected value).

Now I wasn't taught much statistics in school so please forgive me if I am asking something really stupid, I tend to do that.

Basically my question arises from always hearing or reading "If you have a large enough sample", yet no mention of how much is actually a large sample. I made some random number generators and saw the law in action, but my pick for the sample size was around 10000-100000, which was completely random.

One of the comments on the webpage mentioned that the sample size should be the number of options squared divided by the probability of the least probable side, which does not make much sense to me.

Then I thought: The more samples you take, the closer you get to the expected value, but you can take infinite sample and you never actually reach the expected value.

This reminds me of radioactive decay! Both are random, both converge to something and both never fully stop.

I then tried to simplify my question and maybe give it some clear aim point and came up with the following:

How many samples do I need to take to reach the half-life of the convergence? (I assume that it will depend on the probability of the sides of the die roll, or coin toss?)

In other words: How big is a large sample?

P.S. Not sure if my title sounds too similar to some related but not the same question and not sure how to tag this.

Edit: (notes for myself regarding the answer by Ross Millikan)

Sigma levels ($3\sigma$): http://en.wikipedia.org/wiki/Six_Sigma#Sigma_levels

standard deviation = $\sigma = \sqrt{np(1-p)}$
p = probability = 0.5 for coin toss
n = number of trials

To get the number of tosses of a coin required to find the probability of a coin toss with a possible error of $\pm0.1$ with a $3\sigma$ level of accuracy, I would have to do

$3\sigma = 0.1n$

$0.1n = 3\sqrt{np(1−p)}$

$p = 0.5$ for coin toss
$n = (\frac{3}{0.1})^2p(1-p)$
$n = 225$

so I would have to do 225 coin tosses

where the standard deviation $\sigma$ will be
$\sigma = \sqrt{(225)(0.5)(1-0.5)} = 7.5$

$3\sigma = 0.1n = 0.1(225) = 22.5$
expected number of heads and tails is $np = 225*0.5 = 112.5$

As long as we are within 22.5 of 112.5, then the value for p that we get will be $0.5\pm0.1$


If you sample $n$ items that have probability $p$ of having some characteristic ($\frac 12$ for a coin flip, $\frac 16$ for a specific die roll, maybe $\frac 1{10}$ for being under 6 years old) the expected number is $np$. The standard deviation on that number is $\sqrt{np(1-p)}$. If $p$ is not too large we often ignore the factor $(1-p)$ and say the standard deviation is the square root of the number of observed events. If you are trying to measure $p$ with some accuracy, this gives you a measure of how large $n$ has to be. Say you want $p$ within $0.1$ at the $3\sigma$ level. Then you need $0.1n =3\sqrt{np}, n=900p$. So if we sample $450$ coin tosses, we expect $225$ heads with a standard deviation of $15$. As long as we are within $45$ of $225$ we will assess $p$ to be in the range $0.4$ to $0.6$ Since $p$ was fairly large, the standard deviation is actually closer to $11$, but you get the idea.

  • $\begingroup$ if I use the full formula in your example (not ignoring the (1 - p) part), then n = 900(0.5)(1 - 0.5) right? so we would sample 225 coin tosses and expect around 112.5 heads with 7.5 standard deviation, right? $\endgroup$ – Timo Huovinen Dec 2 '14 at 20:58
  • $\begingroup$ The standard deviation would be $\sqrt{900\cdot 0.5(1-0.5)}=15$ $\endgroup$ – Ross Millikan Dec 2 '14 at 22:58
  • $\begingroup$ I don't understand where you got the 900 from, it can't be the same 900 as the one in n=900p because it has a p next to it and the sample size is 450, (it took me almost a year to understand your answer, mainly because I had to learn the definitions first). P.S. how do you do the math like that? $\endgroup$ – Timo Huovinen Dec 3 '14 at 14:15
  • $\begingroup$ Yes, you are correct for $225$ and $7.5$. I had misunderstood. $\endgroup$ – Ross Millikan Dec 3 '14 at 14:21
  • $\begingroup$ Thank you very much for helping out and confirming that I understood it, your answer was more than helpful! $\endgroup$ – Timo Huovinen Dec 3 '14 at 14:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.