# Flipping a coin 1000 times.

Say you have a coin. You don't know the bias of this coin. That it, it flips heads with probability $p$, and tails with $1-p$ for $p \in [0, 1]$.

Somebody flips the coin 1000 times. It comes up heads on every time and then asks you: "What is the bias of this coin"?

You cannot guess it with certainty, as any non-zero $p$ is possible. However, what is the most likely value of $p$?

I'm not sure how to approach this. It seems actually that all values of p have 0 chance of being correct, as there's an infinite number of values $p$ can be. So I thought about $P(p \in [0, 0.1])$, $P(p \in (0.1, 0.2])$ etc. Then finding the group with the highest probability, in this case would be the last one. Then disecting that group into equal parts and starting again, eventually converging on the actual value.

I'm not sure how to formalize this process, or how to actually find $P(p \in [lower, upper])$

-
take a look at the central limit theorem (again?). – mookid Apr 9 '14 at 17:28
@mookid never heard of this theorem, I just thought this up in my head and am curious. I'll read the wiki page when I have time later – Cruncher Apr 9 '14 at 17:29
The maximum likelihood estimation is one way. There you look at what value of $p$ yields the largest value of $P(\text{Your result}\mid\text {Bias}=p)$. In the case of a coin flip, though, it is more accurate the closer to $50-50$ the result is so it might not be the best approach. – Arthur Apr 9 '14 at 17:34
@Cruncher you really can't do much in probability without this theorem and the large law of numbers. – mookid Apr 9 '14 at 17:37
@Cruncher You meant to say $1-p$ instead of $p-1$ – drhab Apr 9 '14 at 17:38

All depends on the a priori probabilities assumed. For simplicity we assume $P_0(p\in[0,x])=x$ for $0\le x\le 1$ as a priori probability (i.e. uniform distribution). For fixed (infinitesimally) small $\epsilon$, then the a posteriori probability of $p\in[x,x+\epsilon]$ is proportional to the probability that 1000 times head occurs if $p\approx x$, which is $x^{1000}$. Thus the density function is proportional to $p^{1000}$, the cumulated is proportional - and as the value for $p=1$ matches in fact identical - to $p^{1001}$. This gives us $$P(p\in[lower,uppre])=upper^{1001}-lower^{1001}.$$

-
$$P(p\in[lower,uppre])=upper^{1001}-lower^{1001}.$$ This seems to suggest that 100% is the most likely value of p. Because For $a, b, c, d | a-b=c-d, a > c$ we get $a^{1001}-b^{1001} > c^{1001}-d^{1001}$ (or at least that seems true to me). So as a result, if we slice the range into any number of equally sized pieces, the highest probability is the last one. – Cruncher Apr 9 '14 at 18:57
Yes. If a coin is experimentally verified to "always" show head, it is not too bad to assume that it always shows head. At least this is correct, if we assume that the a priori distribution of $p$ is uniform. – Hagen von Eitzen Apr 9 '14 at 20:17
Hmm, after a little more thought, I guess that makes sense. I think what was tripping me up is the fact that it's more likely to not be 100% than it is to be 100%. But 100% is still more likely than anything else. And the bias in my head was probably not assigning a uniform distribution to p since most coins are near-fair. Does that mean the answer to this kind of question in general is just the simple: $H / (H+T)$ to get the most likely probability? – Cruncher Apr 10 '14 at 12:42

You can look for the value of $p$ that maximizes the likelihood of the data you have. The likelihood $L$ of $p$ under your data ($H = 1000$) is defined as $L(p|H=1000) = P(H=1000|p) = p^{1000}$. It is obvious that this has its maximal value for $p=1$, because it is increasing in $p$ and $1$ is the maximum possible value for a probability.

You can also take the Bayesian view and assume a probability distribution for $p$. If you take the uniform distribution (the $B(1,1)$ distribution) as your prior – a non-informative prior – and update it according to the Bayesian theorem with the likelihood function, then you get a $B(1+H,1+T)$ distribution ($T$ denotes the number of tails). In your case, it is a $B(1001,1)$ distribution, which has $\frac{1001-1}{1001+1-2} = \frac{1000}{1000} = 1$ as mode. Also in this sense, it is the "most likely value" that you search.

If you want to get the probability in a certain interval, you just have to integrate the density of the $B(1+H,1+T)$ distribution. As stated in another answer, the beta distribution that you get depends on the prior, which is a statement about the information on the coin that you have before conducting the experiment.

-