# What is the standard method to estimate probabilities of events based on observations?

I start from a simple example. I through a (possibly unfair) coin 20 times. I got "eagle" and "tail" 15 and 5 times, respectively. Now I need to estimate probabilities for eagle and tail. For eagle I get 15/20 = 0.75 and for the tail I get 0.25.

However, I know that the probability that I got are not accurate. It is still possible that I have a fair coin (with probability for eagle equal to 0.5) and I get more eagles in my experiment just by chance.

Now I want to estimate probabilities of probabilities. In other words I want to know how likely it is that probability for eagle is 0.5 (or 0.3, or 0.9). I can solve this problem, but I would like to know if there is a name for this problem. I an also interested in the generalization of the problem for the case of more than two events (not just "eagle" and "tail").

-
You are also asking for a parameter's likelihood with respect to the data. – Raphael Nov 26 '10 at 10:35
You would probably also get excellent answers to this question on CrossValidated: stats.stackexchange.com – Jonas Meyer Nov 27 '10 at 4:17

We want to find $p,0\le p\le 1,$ such that $${20\choose 15}p^{15}(1-p)^5$$ is maximum. By simple calculus, this $p$ turns out to be 15/20, as expected.

-

The maximum likelihood estimate is a typical method for estimating the probability distribution given some outcomes. To describe it simply, you assume that your probability distribution you are sampling from is actually one from some set $\Omega$, and you are trying to find which one fits best the data (i.e. which is most likely to be correct).

In your case, you could reasonably assume that the probability distribution $\theta_\pi$ for which "heads" has probability $\pi$ and "eagle" has probability $1-\pi$. So the question is: Which $\pi \in [0,1]$ is most likely to by the actual distribution, given the observed outcomes?