# Deriving the probability of an event from limited trial data

I have a stochastic event (say a weighted coin toss) that produces a positive outcome (heads) according to some unknown probability $P$.

Given $N$ total events (coin tosses) and n positive outcomes (heads), how can I measure the likelihood that $\frac nN$ is a good approximation of $P$?

Obviously, as $N$ grows, it becomes more likely than $\frac nN$ is close to the value $P$, but how can this convergence be described mathematically?

While I realise this isn't a full answer, the convergence you discuss is guaranteed by the weak and strong laws of large numbers. You might also be interested in Chebychev's inequality, which would say $$\mathbb{P}(|n/N - p| > \alpha) \leq \frac{\sigma^2}{\alpha^2}$$ where $\sigma^2$ is the variance of your estimator, $n/N$. Finally the Central Limit Theorem (CLT) proves that as $N \to \infty$ the estimator becomes normally distributed about $p$ with variance $\frac{\sigma^2}{N}$.
The asymptotic theory, when $Np(1-p)$ is very large, is described by the Central Limit Theorem, which says that $n = pN + A\sqrt{Np(1-p)}$ where the probability distribution of $A$ is the famous Gaussian (normal, "bell shaped") curve. Qualitatively, fluctuations of $n$ around the expected value are usually within a small factor of $\sqrt{Np(1-p)}$.