# Coin toss bias estimation problem

Alice and Bob are given a set of five biased coins. They both estimate the probability that each coin will show a head when flipped, and each coin is then flipped once. These are the estimates and values observed:

Coin $1$: Alice's estimate $0.4$, Bob's estimate $0.2$, Observed - Heads

Coin $2$: Alice's estimate $0.7$, Bob's estimate $0.8$, Observed - Heads

Coin $3$: Alice's estimate $0.2$, Bob's estimate $0.3$, Observed - Tails

Coin $4$: Alice's estimate $0.9$, Bob's estimate $0.6$, Observed - Tails

Coin $5$: Alice's estimate $0.4$, Bob's estimate $0.3$, Observed - Heads

Whom would you say is better at estimating the bias of the coins, and why?

I would like guidance on how to approach this question.

• What is the question? Commented Oct 8, 2016 at 10:22
– Hogg
Commented Oct 8, 2016 at 10:24
• What results do you expect for someone who is good at predicting bias? What would you expect from someone who is worse at that? Commented Oct 8, 2016 at 10:31
• I was thinking that the person with the higher expectation would be better, i.e. $E(\text{Alice's estimate of outcome})=0.4+0.7+0.8+0.1+0.4=2.4$, $E(\text{Bob's estimate of outcome})=0.2+0.8+0.7+0.4+0.3=2.4$, but they both add up to the same value.
– Hogg
Commented Oct 8, 2016 at 10:46
• @Hogg - those additions are not probabilities if they exceed $1$. Commented Oct 8, 2016 at 10:49

There are several possibilities. Here is one, which uses multiplication rather than your suggestion of addition:

Alice's estimated likelihood of the observed outcome is $0.4 \times 0.7 \times 0.8 \times 0.1 \times 0.4 = 0.00896$

Bob's estimated likelihood of the observed outcome is $0.2 \times 0.8 \times 0.7 \times 0.4 \times 0.3 = 0.01344$

So Bob's estimated likelihood is $1.5$ times Alice's estimated likelihood, suggesting Bob's estimates may have been slightly better. Alice's estimate for Coin $4$ was particularly unfortunate.

But somebody else who thought there was no predictable bias might say $0.5$ each time and have an overall likelihood for the outcome of $0.5^5=0.03125$, more than double Bob's figure, suggesting that neither were particularly good.

• If you consider the coin toss estimation of Coin $4$ as an anomalous result and ignore it, we have that Alice's estimated likelihood of the observed outcome is $0.4\times 0.7 \times 0.8 \times 0.4=0.0896$ and Bob's estimated likelihood of the observed outcome is $0.2\times 0.8 \times 0.7 \times 0.3=0.0336$, this would make Alice's estimated likelihood $2.67$ times Bob's estimated likelihood.
– Hogg
Commented Oct 8, 2016 at 11:11
• This approach is called Bayesian Model Comparison. See en.wikipedia.org/wiki/Bayes_factor Commented Oct 8, 2016 at 13:54

One approach is to look at the sum of the squared deviations from the observed outcome ($1$ if heads, $0$ if tails):

$$A: \sqrt{0.6^2+0.3^2+0.2^2+0.9^2+0.6^2}= 1.288\dots$$ $$B: \sqrt{0.8^2+0.2^2+0.3^2+0.6^2+0.7^2}=1.273\dots$$

So, Bob wins, but not significantly!

Another is to count who came the closest the most times, which results in $3:2$ in favor of Alice. This, however, has the same "drawbacks" as for instance the scoring system in Tennis, where one player can actually win more games (which for our example would be equivalent to being closer overall), but still loose the match.

• How about squaring the error? The error would go down, so maybe square rooting the abs? Commented Oct 8, 2016 at 13:52
• @OFRBG Thanks for the suggestion, edited! Commented Oct 8, 2016 at 13:55