Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have a stochastic event (say a weighted coin toss) that produces a positive outcome (heads) according to some unknown probability $P$.

Given $N$ total events (coin tosses) and n positive outcomes (heads), how can I measure the likelihood that $\frac nN$ is a good approximation of $P$?

Obviously, as $N$ grows, it becomes more likely than $\frac nN$ is close to the value $P$, but how can this convergence be described mathematically?

share|cite|improve this question
up vote 3 down vote accepted

While I realise this isn't a full answer, the convergence you discuss is guaranteed by the weak and strong laws of large numbers. You might also be interested in Chebychev's inequality, which would say $$\mathbb{P}(|n/N - p| > \alpha) \leq \frac{\sigma^2}{\alpha^2}$$ where $\sigma^2$ is the variance of your estimator, $n/N$. Finally the Central Limit Theorem (CLT) proves that as $N \to \infty$ the estimator becomes normally distributed about $p$ with variance $\frac{\sigma^2}{N}$.

share|cite|improve this answer

This is the problem of confidence intervals in statistics. Your question is about the original problem of that type, the "confidence interval for a proportion in binomial sampling" (wikipedia )

The asymptotic theory, when $Np(1-p)$ is very large, is described by the Central Limit Theorem, which says that $n = pN + A\sqrt{Np(1-p)}$ where the probability distribution of $A$ is the famous Gaussian (normal, "bell shaped") curve. Qualitatively, fluctuations of $n$ around the expected value are usually within a small factor of $\sqrt{Np(1-p)}$.

For limited data, the story is more complicated, with no perfect solution, many proposals, and a large literature.

share|cite|improve this answer

The wiki on convergence of random variables is what you probably want to read. It discusses various forms of convergence and in particular convergence in probability and almost sure convergence are what you need to take a look at.

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.