# Deriving the probability of an event from limited trial data

I have a stocastic event (say a weighted coin toss) that produces a positive outcome (heads) according to some unknown probability P.

Given N total events (coin tosses) and n positive outcomes (heads), how can I measure the likelihood that n/N is a good approximation of P?

Obviously, as N grows, it becomes more likely than n/N is close to the value P, but how can this convergence be described mathematically?

-

While I realise this isn't a full answer, the convergence you discuss is guaranteed by the weak and strong laws of large numbers. You might also be interested in Chebychev's inequality, which would say $$\mathbb{P}(|n/N - p| > \alpha) \leq \frac{\sigma^2}{\alpha^2}$$ where $\sigma^2$ is the variance of your estimator, $n/N$. Finally the Central Limit Theorem (CLT) proves that as $N \to \infty$ the estimator becomes normally distributed about $p$ with variance $\frac{\sigma^2}{N}$.

-

This is the problem of confidence intervals in statistics. Your question is about the original problem of that type, the "confidence interval for a proportion in binomial sampling" (wikipedia http://en.wikipedia.org/wiki/Binomial_proportion_confidence_interval )

The asymptotic theory, when $Np(1-p)$ is very large, is described by the Central Limit Theorem, which says that $n = pN + A\sqrt{Np(1-p)}$ where the probability distribution of $A$ is the famous Gaussian (normal, "bell shaped") curve. Qualitatively, fluctuations of $n$ around the expected value are usually within a small factor of $\sqrt{Np(1-p)}$.

For limited data, the story is more complicated, with no perfect solution, many proposals, and a large literature.

-