Coin tosses with unknown success probability

Question

Suppose I have a coin with unknown probability of success (let say Heads) $p$ which is uniformly distributed on $[0, 1]$.

I toss a coin $N$ times.

What is the probability that I have got $n \leq N$ Heads?

Ok. I've calculated (using Wolfram) that it is exactly $\frac{1}{N+1}$, i.e. every $n$ is equiprobable.

What is the intuitive explanation for this fact?

I can understand by symmetry argument, why getting $k$ Heads should be exactly the same as getting $k$ Tails. But why getting $0$ Heads is the same as getting $7$ Heads (in case $N \neq 7$) is well beyond me.

Update: Calculations involved: $$P(\text{# of Heads} = k) = \int\limits_0^1 {N \choose k} p^k (1-p)^{N-k} dp, $$

solving it gives: $$P(\text{# of Heads} = k) = {N \choose k} \frac{\Gamma(k+1)\Gamma(N-k+1)}{\Gamma(N+2)},$$

which turns out to be $\frac{1}{N+1}$.

Answer 1

Of course there is an intuition.

To see it let's ask the question differently. For that I will be considering $X_0, X_1,\ldots,X_{N}$ as $N+1$ independent uniform distributed random variables on $(0,1)$.

$X_0$ will be the probability of having heads.

The number of heads in your trials is exactly the number of $X_i$ smaller than $X_0$. So if you want to have $k$ heads you want $X_0$ to be ranked as the $k+1$-th element.

However there is no difference between $X_0$ and any other $X_j$. So ranking $X_0$ as the $k+1$-th element will have $1/(N+1)$ probability.

Answer 2

Ok. I've calculated (using Wolfram) that it is exactly $\frac{1}{N+1}$, i.e. every $n$ is equiprobable.

That result is correct, but we don't need no stinkin' Wolfram Alpha to do this.

$$P(\text{# of Heads} = k) = \int\limits_0^1 {N \choose k} p^k (1-p)^{N-k} dp, $$

Well first of all $$ \int\limits_0^1 {N \choose k} p^k (1-p)^{N-k} dp ={N \choose k}\int_0^1 p^k(1-p)^{N-k} dp $$ Now the integral $Q(k,N)\equiv\int_0^1 p^k(1-p)^{N-k}dp$ can be done e.g. by integration by parts. If $k>0$ then $$ \begin{align} \int p^k(1-p)^{N-k}&=p^k\int (1-p)^{N-k}-k\int p^{k-1}\int(1-p)^{N-k+1}\\ &={1\over N-k+1}p^k(1-p)^{N-k+1}\biggr|_0^1\\ &\;\;\;\;\;\;+{k\over N-k+1}\int_0^1p^{k-1}(1-p)^{N-k+1}dp\\ &=0+{k\over N-k+1}Q(k-1,N) \tag{1} \end{align} $$ and if $k=0$ then $$Q(0,N)=\int_0^1 (1-p)^N dp={1\over N+1}\tag{2}$$ so using (1) and (2) $$Q(k,N)={k!(N-k)!\over (N+1)!}={1\over(N+1) {N\choose k}}.$$

What is the intuitive explanation for this fact?

The obvious intuitive explanation for the probabilities all being ${1\over N+1}$ is that when we have no idea whatsoever what the probability of getting a head is on any trial, that is to say "there is a uniform distribution of probability of getting a head", then we have absolutely no idea whatsoever what outcome we will get on each trial, so the probability of getting $k$ heads on $N$ trials comes out to be exactly the same for all the different $k$s.

It's quite neat that the maths comes out like this.

Answer 3

To be more explicit, what you are describing can be viewed in conditional probability with an uninformative uniform prior on $p$.

That is, first draw $p \sim U[0,1]$, then draw $k \mid p \sim \operatorname{Bin} (N,p)$.

As you are aware, conditional probability tells us

$$ P(k) = \int P(k \mid p) P(p) dp $$

The other answers give good intuition: the uninformative prior in this case gives "no information" via $p$ of $k$, and it turns out in this case due to some mathematical symmetries of binomial distribution (the sum of independent Bernoulli trials) that if we know "nothing" about $p$ then we also know "nothing" about $k$ so it comes out as uniform.

And if I encountered this problem in real life and wanted to mathematically verify my intuition, there's actually a shortcut statisticians use to avoid any integrals:

Recall that Binomial likelihood with Beta conjugate prior (where continuous uniform distribution $U(0,1)$ is a special case $\operatorname{Beta}(1,1)$) has a beta-binomial posterior-predictive (marginalizing over $p$), plug-in $\alpha = \beta = 1$, and the discrete uniform distribution $U(0, N)$ again pops out :)