# Coin tosses with unknown success probability

Suppose I have a coin with unknown probability of success (let say Heads) $$p$$ which is uniformly distributed on $$[0, 1]$$.

I toss a coin $$N$$ times.

What is the probability that I have got $$n \leq N$$ Heads?

Ok. I've calculated (using Wolfram) that it is exactly $$\frac{1}{N+1}$$, i.e. every $$n$$ is equiprobable.

What is the intuitive explanation for this fact?

I can understand by symmetry argument, why getting $$k$$ Heads should be exactly the same as getting $$k$$ Tails. But why getting $$0$$ Heads is the same as getting $$7$$ Heads (in case $$N \neq 7$$) is well beyond me.

Update: Calculations involved: $$P(\text{# of Heads} = k) = \int\limits_0^1 {N \choose k} p^k (1-p)^{N-k} dp,$$

solving it gives: $$P(\text{# of Heads} = k) = {N \choose k} \frac{\Gamma(k+1)\Gamma(N-k+1)}{\Gamma(N+2)},$$

which turns out to be $$\frac{1}{N+1}$$.

• Huh? What is your calculation? What is the prior? Is it the uniform prior? Aug 16 at 19:35
• @BrianTung, since you ask and I do not understand you, we can assume that indeed uniform. Aug 16 at 19:36
• What I mean is, are you assuming that $p$ is uniformly distributed a priori? I.e., before you start flipping, $p$ is equally likely to be any value between $0$ and $1$? Aug 16 at 19:46
• @DavidG.Stork There's only symmetry if $N=7$, which I don't think is assumed. Aug 17 at 4:59
• @Youem With an argument such as the one you've given, yes! I'm just saying that an argument like that needs to be made; the $k=0$ case is only obviously symmetric to the $k=N$ case. Aug 17 at 5:21

Of course there is an intuition.

To see it let's ask the question differently. For that I will be considering $$X_0, X_1,\ldots,X_{N}$$ as $$N+1$$ independent uniform distributed random variables on $$(0,1)$$.

$$X_0$$ will be the probability of having heads.

The number of heads in your trials is exactly the number of $$X_i$$ smaller than $$X_0$$. So if you want to have $$k$$ heads you want $$X_0$$ to be ranked as the $$k+1$$-th element.

However there is no difference between $$X_0$$ and any other $$X_j$$. So ranking $$X_0$$ as the $$k+1$$-th element will have $$1/(N+1)$$ probability.

• That is a good argument, I read it once and didn't understand it, but reading it again I think it's a very good way to look at this. Aug 17 at 1:49
• So you say: "Let first toss a probability of success", then "count as success everything that is tossed in range $[0, X_1]$". I like it for simplicity, but why is this a different view on the coin tosses? This is unclear. Aug 17 at 5:22
• Toss a coin with success probability $p$ is equivalent to generate a random variable uniform on $(0,1)$ and see if it smaller than $p$ Aug 17 at 5:58
• It does need re-reading. My thinking: Let $H=\sum\limits_{i=2}^{N+1} \mathbf{1}_{X_i<X_1}$ be the resulting number of heads. We have $\mathbb{P}(H=h \mid X_1=p) ={N \choose h}p^h(1-p)^{N-h}$ as a conditional binomial distribution, while $f_{X_1}(x)=1$ for $0 \le x \le 1$ and marginally $\mathbb{P}(H=h) =\frac{1}{N+1}$ with integer $0 \le h \le N$ due to the exchangeability. It might have been easier if the random variables had been $X_0,X_1,\ldots,X_N$ Aug 17 at 8:35
• Youem - as @SuzuHirose commented, it is not instantly obvious to me what you said, so I reread it and then wrote those notes to help me understand what you were saying. For me, the intuitive bit is that you do not need to calculate the marginal $\int_0^1 {N \choose h}p^h(1-p)^{N-h} \, dp$ since your model makes it clear it must be $\frac1{N+1}$. I am happy to leave this in the comments, though I have edited your answer to make the special value $X_0$ Aug 17 at 12:57

Ok. I've calculated (using Wolfram) that it is exactly $$\frac{1}{N+1}$$, i.e. every $$n$$ is equiprobable.

That result is correct, but we don't need no stinkin' Wolfram Alpha to do this.

$$P(\text{# of Heads} = k) = \int\limits_0^1 {N \choose k} p^k (1-p)^{N-k} dp,$$

Well first of all $$\int\limits_0^1 {N \choose k} p^k (1-p)^{N-k} dp ={N \choose k}\int_0^1 p^k(1-p)^{N-k} dp$$ Now the integral $$Q(k,N)\equiv\int_0^1 p^k(1-p)^{N-k}dp$$ can be done e.g. by integration by parts. If $$k>0$$ then \begin{align} \int p^k(1-p)^{N-k}&=p^k\int (1-p)^{N-k}-k\int p^{k-1}\int(1-p)^{N-k+1}\\ &={1\over N-k+1}p^k(1-p)^{N-k+1}\biggr|_0^1\\ &\;\;\;\;\;\;+{k\over N-k+1}\int_0^1p^{k-1}(1-p)^{N-k+1}dp\\ &=0+{k\over N-k+1}Q(k-1,N) \tag{1} \end{align} and if $$k=0$$ then $$Q(0,N)=\int_0^1 (1-p)^N dp={1\over N+1}\tag{2}$$ so using (1) and (2) $$Q(k,N)={k!(N-k)!\over (N+1)!}={1\over(N+1) {N\choose k}}.$$

What is the intuitive explanation for this fact?

The obvious intuitive explanation for the probabilities all being $${1\over N+1}$$ is that when we have no idea whatsoever what the probability of getting a head is on any trial, that is to say "there is a uniform distribution of probability of getting a head", then we have absolutely no idea whatsoever what outcome we will get on each trial, so the probability of getting $$k$$ heads on $$N$$ trials comes out to be exactly the same for all the different $$k$$s.

It's quite neat that the maths comes out like this.

To be more explicit, what you are describing can be viewed in conditional probability with an uninformative uniform prior on $$p$$.

That is, first draw $$p \sim U[0,1]$$, then draw $$k \mid p \sim \operatorname{Bin} (N,p)$$.

As you are aware, conditional probability tells us

$$P(k) = \int P(k \mid p) P(p) dp$$

The other answers give good intuition: the uninformative prior in this case gives "no information" via $$p$$ of $$k$$, and it turns out in this case due to some mathematical symmetries of binomial distribution (the sum of independent Bernoulli trials) that if we know "nothing" about $$p$$ then we also know "nothing" about $$k$$ so it comes out as uniform.

And if I encountered this problem in real life and wanted to mathematically verify my intuition, there's actually a shortcut statisticians use to avoid any integrals:

Recall that Binomial likelihood with Beta conjugate prior (where continuous uniform distribution $$U(0,1)$$ is a special case $$\operatorname{Beta}(1,1)$$) has a beta-binomial posterior-predictive (marginalizing over $$p$$), plug-in $$\alpha = \beta = 1$$, and the discrete uniform distribution $$U(0, N)$$ again pops out :)

• Fantastic answer, motivating the construction of the random variable over and beyond the result, and I think it gives a very decent and general explanation of the situation. Great stuff right here. Aug 17 at 10:40