8
$\begingroup$

Suppose I have a coin with unknown probability of success (let say Heads) $p$ which is uniformly distributed on $[0, 1]$.

I toss a coin $N$ times.

What is the probability that I have got $n \leq N$ Heads?

Ok. I've calculated (using Wolfram) that it is exactly $\frac{1}{N+1}$, i.e. every $n$ is equiprobable.

What is the intuitive explanation for this fact?

I can understand by symmetry argument, why getting $k$ Heads should be exactly the same as getting $k$ Tails. But why getting $0$ Heads is the same as getting $7$ Heads (in case $N \neq 7$) is well beyond me.

Update: Calculations involved: $$P(\text{# of Heads} = k) = \int\limits_0^1 {N \choose k} p^k (1-p)^{N-k} dp, $$

solving it gives: $$P(\text{# of Heads} = k) = {N \choose k} \frac{\Gamma(k+1)\Gamma(N-k+1)}{\Gamma(N+2)},$$

which turns out to be $\frac{1}{N+1}$.

$\endgroup$
10
  • 2
    $\begingroup$ Huh? What is your calculation? What is the prior? Is it the uniform prior? $\endgroup$
    – Brian Tung
    Commented Aug 16, 2022 at 19:35
  • $\begingroup$ @BrianTung, since you ask and I do not understand you, we can assume that indeed uniform. $\endgroup$
    – dEmigOd
    Commented Aug 16, 2022 at 19:36
  • 1
    $\begingroup$ What I mean is, are you assuming that $p$ is uniformly distributed a priori? I.e., before you start flipping, $p$ is equally likely to be any value between $0$ and $1$? $\endgroup$
    – Brian Tung
    Commented Aug 16, 2022 at 19:46
  • 1
    $\begingroup$ @DavidG.Stork There's only symmetry if $N=7$, which I don't think is assumed. $\endgroup$ Commented Aug 17, 2022 at 4:59
  • 2
    $\begingroup$ @Youem With an argument such as the one you've given, yes! I'm just saying that an argument like that needs to be made; the $k=0$ case is only obviously symmetric to the $k=N$ case. $\endgroup$ Commented Aug 17, 2022 at 5:21

3 Answers 3

13
$\begingroup$

Of course there is an intuition.

To see it let's ask the question differently. For that I will be considering $X_0, X_1,\ldots,X_{N}$ as $N+1$ independent uniform distributed random variables on $(0,1)$.

$X_0$ will be the probability of having heads.

The number of heads in your trials is exactly the number of $X_i$ smaller than $X_0$. So if you want to have $k$ heads you want $X_0$ to be ranked as the $k+1$-th element.

However there is no difference between $X_0$ and any other $X_j$. So ranking $X_0$ as the $k+1$-th element will have $1/(N+1)$ probability.

$\endgroup$
7
  • 4
    $\begingroup$ That is a good argument, I read it once and didn't understand it, but reading it again I think it's a very good way to look at this. $\endgroup$ Commented Aug 17, 2022 at 1:49
  • 1
    $\begingroup$ So you say: "Let first toss a probability of success", then "count as success everything that is tossed in range $[0, X_1]$". I like it for simplicity, but why is this a different view on the coin tosses? This is unclear. $\endgroup$
    – dEmigOd
    Commented Aug 17, 2022 at 5:22
  • 1
    $\begingroup$ Toss a coin with success probability $p$ is equivalent to generate a random variable uniform on $(0,1)$ and see if it smaller than $p$ $\endgroup$
    – Kroki
    Commented Aug 17, 2022 at 5:58
  • $\begingroup$ It does need re-reading. My thinking: Let $H=\sum\limits_{i=2}^{N+1} \mathbf{1}_{X_i<X_1}$ be the resulting number of heads. We have $\mathbb{P}(H=h \mid X_1=p) ={N \choose h}p^h(1-p)^{N-h}$ as a conditional binomial distribution, while $f_{X_1}(x)=1$ for $0 \le x \le 1$ and marginally $\mathbb{P}(H=h) =\frac{1}{N+1}$ with integer $0 \le h \le N$ due to the exchangeability. It might have been easier if the random variables had been $X_0,X_1,\ldots,X_N$ $\endgroup$
    – Henry
    Commented Aug 17, 2022 at 8:35
  • 1
    $\begingroup$ Youem - as @SuzuHirose commented, it is not instantly obvious to me what you said, so I reread it and then wrote those notes to help me understand what you were saying. For me, the intuitive bit is that you do not need to calculate the marginal $\int_0^1 {N \choose h}p^h(1-p)^{N-h} \, dp$ since your model makes it clear it must be $\frac1{N+1}$. I am happy to leave this in the comments, though I have edited your answer to make the special value $X_0$ $\endgroup$
    – Henry
    Commented Aug 17, 2022 at 12:57
4
$\begingroup$

Ok. I've calculated (using Wolfram) that it is exactly $\frac{1}{N+1}$, i.e. every $n$ is equiprobable.

That result is correct, but we don't need no stinkin' Wolfram Alpha to do this.

$$P(\text{# of Heads} = k) = \int\limits_0^1 {N \choose k} p^k (1-p)^{N-k} dp, $$

Well first of all $$ \int\limits_0^1 {N \choose k} p^k (1-p)^{N-k} dp ={N \choose k}\int_0^1 p^k(1-p)^{N-k} dp $$ Now the integral $Q(k,N)\equiv\int_0^1 p^k(1-p)^{N-k}dp$ can be done e.g. by integration by parts. If $k>0$ then $$ \begin{align} \int p^k(1-p)^{N-k}&=p^k\int (1-p)^{N-k}-k\int p^{k-1}\int(1-p)^{N-k+1}\\ &={1\over N-k+1}p^k(1-p)^{N-k+1}\biggr|_0^1\\ &\;\;\;\;\;\;+{k\over N-k+1}\int_0^1p^{k-1}(1-p)^{N-k+1}dp\\ &=0+{k\over N-k+1}Q(k-1,N) \tag{1} \end{align} $$ and if $k=0$ then $$Q(0,N)=\int_0^1 (1-p)^N dp={1\over N+1}\tag{2}$$ so using (1) and (2) $$Q(k,N)={k!(N-k)!\over (N+1)!}={1\over(N+1) {N\choose k}}.$$

What is the intuitive explanation for this fact?

The obvious intuitive explanation for the probabilities all being ${1\over N+1}$ is that when we have no idea whatsoever what the probability of getting a head is on any trial, that is to say "there is a uniform distribution of probability of getting a head", then we have absolutely no idea whatsoever what outcome we will get on each trial, so the probability of getting $k$ heads on $N$ trials comes out to be exactly the same for all the different $k$s.

It's quite neat that the maths comes out like this.

$\endgroup$
4
$\begingroup$

To be more explicit, what you are describing can be viewed in conditional probability with an uninformative uniform prior on $p$.

That is, first draw $p \sim U[0,1]$, then draw $k \mid p \sim \operatorname{Bin} (N,p)$.

As you are aware, conditional probability tells us

$$ P(k) = \int P(k \mid p) P(p) dp $$

The other answers give good intuition: the uninformative prior in this case gives "no information" via $p$ of $k$, and it turns out in this case due to some mathematical symmetries of binomial distribution (the sum of independent Bernoulli trials) that if we know "nothing" about $p$ then we also know "nothing" about $k$ so it comes out as uniform.

And if I encountered this problem in real life and wanted to mathematically verify my intuition, there's actually a shortcut statisticians use to avoid any integrals:

Recall that Binomial likelihood with Beta conjugate prior (where continuous uniform distribution $U(0,1)$ is a special case $\operatorname{Beta}(1,1)$) has a beta-binomial posterior-predictive (marginalizing over $p$), plug-in $\alpha = \beta = 1$, and the discrete uniform distribution $U(0, N)$ again pops out :)

$\endgroup$
1
  • 3
    $\begingroup$ Fantastic answer, motivating the construction of the random variable over and beyond the result, and I think it gives a very decent and general explanation of the situation. Great stuff right here. $\endgroup$ Commented Aug 17, 2022 at 10:40

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .