Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

A friend of mine asked me the following question, and I am not sure how to solve it:

You are given two weighted coins, $C_1$ and $C_2$. Coin $C_1$ has probability $p_1$ of landing heads and $C_2$ has probability $p_2$ of landing heads. The following experiment is preformed:

Coin $C_1$ is flipped 3 times, and lands heads 3 times.

Coin $C_2$ is flipped 10 times, and lands heads 7 times.

Based on this experiment, choose the coin which is more likely to have a higher probability of being heads. In other words, which is more likely: $p_1>p_2$ or $p_2>p_1$.

Intuition tells me coin $C_1$ is the better choice, but this could be wrong, and I am wondering how do you solve this in general. Consider the experiment, $C_1$ is flipped $n_1$ times and lands heads $m_1$ times, $C_2$ is flipped $n_2$ times and lands heads $m_2$ times.

Thanks for the help,

Edit: I think this might answer some questions: Suppose that the probabilities of the coins, $p_1$ and $p_2$ are chosen uniformly from $[0,1]$.

share|improve this question
You really need a prior distribution for how "weighted" the coins may be, and then use Bayesian techniques. –  Henry Apr 18 '11 at 0:18
I would be very interested in seeing a "prior-free", possibly frequentist, approach to this problem, if there is such a thing. I feel like Bayesian techniques get all the press these days. –  Rahul Apr 18 '11 at 0:20
I wonder if you can do this with nonparameteric statistics. The only test I can think of requires that the two sequences have the same length, i.e. $n_1=n_2$ and that's pretty dull here. –  Carl Brannen Apr 18 '11 at 0:49
@Rahul: The question "which is more likely: $p_1 > p_2$ or $p_2 > p_2$" cries out for a solution which treats both as uncertain. –  Henry Apr 18 '11 at 1:15
@Eric: Yes - you get a figure of about 0.758 for the chance $p_1 > p_2$ –  Henry Apr 18 '11 at 8:58
show 1 more comment

3 Answers

up vote 1 down vote accepted

Given a specific probability $p$ of heads, the probability of getting $h$ heads and $t$ tails is just the binomial distribution: $P(H = h, T = t | n, p) = p^h (1-p)^t {n \choose h}$ (though we consider $p$ as varying, rather than $n$ and $h$.

With a uniform prior $g(p) = 1$, that probability is just the weight. The integral of this is $\frac{1}{1 + h + t}$, giving a probability density of $(1 + h + t) p^h (1-p)^t {n \choose h}$. The mean is the integral of $p$ times this, is $(1 + h + t) \int p^{(h+1)} (1-p)^t {n \choose h} dp$. Reusing the result from last time, we know that this must be $(1+h+t) {n \choose h}/{n+1 \choose h + 1}/(2+h+t) = (1+n)(h+1)/(n+1)(n+2) = (h+1)/(n+2)$. This is slightly "hedged toward the center" from the naïve estimator $h/n$ (which is the peak of the distribution).

Another common prior that a Bayesian might use is the beta distribution. It's handy because it is a conjugate prior for the binomial distribution. After collecting data generated by the binomial distribution, the probability is still in the form of a beta distribution. In fact, the uniform prior is just the beta distribution with $\alpha = \beta = 1$. Heads and tail each just add one to the parameters $\alpha$ and $\beta$ respectively. The integrals were essentially worked out above -- factorials generalize to $\Gamma(x+1)$. It's often considered that this case of $\alpha = \beta = 1$ is too conservative, and that $\alpha = \beta = 1/2$ "assumes less" and "lets the data speak more".

With the Uniform Prior (Beta(1,1)), $\overline{p} = (h+1)/(n+2)$:
$C_1$, 3 heads, 0 tails: $\overline{p} = 4/5$
$C_2$, 7 heads, 3 tails: $\overline{p} = 8/12 = 2/3$

$C_1$ is expected to do better.

With the Beta(1/2, 1/2) prior, $\overline{p} = (h+1/2)/(n+1) = (2h+1)/(2n+2)$:
$C_1$, 3 heads, 0 tails: $\overline{p} = 7/8$
$C_2$, 7 heads, 3 tails: $\overline{p} = 15/22$

$C_1$ is expected to do better.

Actually calculating the chances of $C_1$ being better than $C_2$ involve a rather nasty integral, but the calculated $p$ value is enough to tell you which is the better bet.

share|improve this answer
How do you prove that it's ok to compare estimators for $p_1$ and $p_2$ as you did here, instead of actually testing (or finding the probability of) the hypothesis that $p_1 > p_2$? It's intuitive, but does it not require proof? –  ShreevatsaR Jun 2 '11 at 3:26
Huh, you're right @ShreevatsaRt. This only gets $E[p_1 - p_2] > 0$, not $P(p_1 > p_2) > 1/2$. For most purposes what you want is the first, as payoffs are linear in $p$. –  wnoise Jun 2 '11 at 13:45
add comment

As Henry mentions, I think one needs some information about the prior distributions of the weights.

Denote by $r$ the weight of a coin. Suppose that the weight has some prior distribution $g(r)$. Let $f(r|H=h, T=t)$ be the posterior probability density function of $r$ having observed $h$ heads and $t$ tails tossed. Bayes' Theorem tells us that: $$ f(r|H=h, T=t) = \frac{Pr(H=h|r, N = h+t)g(r)}{\int_0^1 Pr(H=h|r, N = h+t)g(r)\ dr}. $$

This should allow you to answer your question. Once you have the posterior pdf for each coin, just find their respective expected weights.

share|improve this answer
add comment

You could do the integration which wnoise is talking about approximately using the following R code, and it is easily adapted to other cases:

> n <- 1000000                      # number of cases to simulate for integration
> prior  <- c(1,1)                  # Beta(1,1) is uniform prior
> coin_1 <- c(3,0)                  # number of heads and tails observed
> coin_2 <- c(7,3)                  # number of heads and tails observed
> p_1    <- rbeta(n, prior[1]+coin_1[1], prior[2]+coin_1[2])
> p_2    <- rbeta(n, prior[1]+coin_2[1], prior[2]+coin_2[2])
> p_diff <- p_1 - p_2
> length(p_diff[p_diff > 0]) / n    # proportion with p_1 > p_2
[1] 0.758118
share|improve this answer
add comment

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.