Here is the sentence which I'm confused with

Imagine I give you a coin, and want you to estimate the probability it will spin to heads. Given what you know, the most reasonable prior belief is to expect any probability of the coin spinning to heads. This can be captured in a uniform prior on pp, the probability that a coin when spun will land on heads: var p = uniform(0,1).

You conduct an experiment. You spin the coin 20 times. 15 of them, they spin to heads.

I'm confused with the part uniform(0,1).
why flipping coin prior can be represent with Uniform(0,1)? I found some related answer here


If random variables (r.v.s) $a_1,a_2,\ldots$ are i.i.d. uniformly distributed on the set $\{0,1\}$, then $\alpha=(0.a_1a_2\ldots)_2$ is a r.v. uniformly distributed on the real interval $[0,1]$. To see this, note that for any $x=(0.x_1x_2\ldots)_2\in[0,1)$ (always taking, WLOG, the unique binary representation of $x$ that has infinitely many $0$s), we have the following: $$\begin{align}\{\alpha > x\} = & \{a_1>x_1\}\cup\\ > &\{\{a_1=x_1\}\cap \{a_2>x_2\}\}\cup\\ &\{\{a_1=x_1\}\cap > \{a_2=x_2\}\cap\{a_3>x_3\} \}\cup\\ &\ldots \end{align}$$

Now, $P(a_i >x_i) = \frac{1}{2}(1-x_i)$, so the probability of the above disjoint union is just $$\begin{align}P(\alpha>x) &= > \frac{1}{2}(1-x_1) + \frac{1}{2^2}(1-x_2) + > \frac{1}{2^3}(1-x_3)+\ldots\\ &= \sum_{i=1}^\infty \frac{1}{2^i} - > \sum_{i=1}^\infty \frac{x_i}{2^i}\\ &= 1 - x\\ \therefore P(\alpha\le > x) &= x \end{align} $$ therefore $\alpha$ is a r.v. uniformly distributed on $[0,1]$.

But I'm quite confused with what is $$P(\alpha > x )$$ and why does he concluded $$P(\alpha \le x)=x$$ is uniform distribution on [0,1]

  • 1
    $\begingroup$ This seems to be a problem given some weighted coin (i.e. you don't know what the probabilities of it landing heads is). This means that any number between $0$ (will never happen) or $1$ (will always happen) is reasonable. Since you want to model this 'belief' about what the probability of heads is then you can model it with a uniform distribution between these values until you can experiment and thus adjust your assumptions about the coin. $\endgroup$ – Twis7ed Apr 13 '17 at 4:04
  • $\begingroup$ If $X \sim \mathsf{Unif}(0,1)$ then the CDF is $P(X \le x) = F_X(x) = x.$ for $0 < x < 1,$ and the PDF is $f_X(x) = 1,$ for $0 < x < 1$ (and $f_X(x) = 0$ elsewhere). $\endgroup$ – BruceET Apr 13 '17 at 7:27

I believe this is intended to be an elementary Bayesian inference problem. It seems you have decided to let $\theta = P(\text{Heads})$ have the "flat" or "noninformative" prior distribution $\mathsf{Unif}(0,1) \equiv \mathsf{Beta}(\alpha=1,\beta=1).$ So your prior distribution is $p(\theta) = 1.$

A uniform prior is not the only possible choice: (a) If you had some prior experience with or knowledge of the coin, you might choose a prior distribution that reflects your prior opinion. Perhaps you look at the coin, play with it a bit, and decide it seems close to fairly balanced. Then you might choose the prior distribution $\mathsf{Beta}(5,5)$ which puts roughly half of its probability in $(.4, .6)$ [based on a computation in R, where qbeta(c(.25,.75), 5, 5) returns 0.3919 and 0.6080.] (b) Another popular choice for a noninformative prior is $\mathsf{Beta}(.5, .5).$ [Google Jeffrey's priors.] Your choice of a prior distribution will influence your final conclusions after you have data and get a posterior distribution. (I think @Twis7ed's Comment is suggesting that priors other than uniform are possible.)

Then from your experiment, you have $n=20$ Bernoulli trials resulting in $x = 15$ Heads, so your likelihood function is $p(x|\theta) = \theta^x(1-\theta)^{n-x} = \theta^{15}(1-\theta)^5.$

Then according to Bayes' Theorem, the posterior distribution (using the uniform prior) is $$p(\theta|x) \propto p(\theta) \times p(x|\theta) \propto \theta^{15}(1-\theta)^5,$$

which we recognize as the kernel (PDF without constant) of $\mathsf{Beta}(16,6).$

The posterior distribution can be used to get a point estimate, which might be the posterior mean $\frac{16}{16+6} = 0.7272$, posterior median 0.7343 (from R), or posterior mode $\frac{15}{20} = 0.75.$

qbeta(.5, 16, 6)
[1] 0.7342603

You could also use the posterior distribution to find a Bayesian 95% probability interval $(0.53, 0.89)$ (from R).

qbeta(c(.025, .975), 16, 6)
[1] 0.5283402 0.8871906

Note: Maybe you can get the posterior distribution using the prior $\mathsf{Beta}(5, 5)$ and the same likelihood function, and see what difference that alternate choice of prior distribution makes in the point and interval estimates.


With a typical coin it is quite unreasonable to assign a uniform distribution on the interval from $0$ to $1$ as the probability distribution of the frequency with which "heads" appear. For example, the uniform distribution would mean that there is a $10\%$ chance that "heads" appear more than $90\%$ of the time and there is a $10\%$ chance that "heads" appear less than $10\%$ of the time. That is not reasonable as applied to any normal coin. So you have to take it as a given in the math problem that is set here.

The physicist Edwin Jaynes has argued that this is a reasonable prior when you know that "heads" and "tails" are both possible outcomes but you know nothing else. That is certainly different from the situation with most coins.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.