# How would you approach this problem on the Bayes theorem?

I've been reading a book on Statistics and I could COMPLETELY understand all of its text. It basically explained the bayes theorem and what priors were, what posteriors were etc. But then in the exercise I came across this problem:

There are 3 possible values for µ. It could be fair µ =1/2, biased tails µ =1/4, or biased heads µ =3/4.

Assume our prior is that each of these possibilities is equally likely. Looking forward, what is the minimum number of tosses we’d need to see in order to conclude that p(µ =1/2) >1/2 (strictly greater)? Give an example of a sequence of tosses with this length which would lead to this conclusion. Prove your results.

And I have no clue at all how to approach this.

In addition, what is the minimum number of tosses we’d need to see in order to conclude that p(µ =3/4) >1/2 (strictly greater)? Give an example of a sequence of tosses with this length which would lead to this conclusion. Prove your results.

-
– Dilip Sarwate Feb 13 '13 at 4:14
@DilipSarwate is it frowned upon? – Programming Noob Feb 13 '13 at 4:15
@ProgrammingNoob Yes indeed. Please close one of the questions, or ask the moderators of the site to move it to the other site. You can contact the moderators by clicking on the flag link at the bottom of your question, above the comments. – Dilip Sarwate Feb 13 '13 at 4:17

The probability of getting $k$ heads and $l$ tails given $\mu$ is

$$\binom{k+l}k\mu^k(1-\mu)^l\;.$$

Thus the probability for $\mu=\frac12$ given $k$ heads and $l$ tails is

$$\frac{\left(\frac12\right)^k\left(\frac12\right)^l}{\left(\frac14\right)^k\left(\frac34\right)^l+\left(\frac12\right)^k\left(\frac12\right)^l+\left(\frac34\right)^k\left(\frac14\right)^l}\;.$$

Given $k+l$, this is maximal when $\left(\frac14\right)^k\left(\frac34\right)^l+\left(\frac34\right)^k\left(\frac14\right)^l$ is minimal, and thus when $3^l+3^k$ is minimal, and this happens when $k$ and $l$ are as equal as possible, as one would expect. Thus for given $n=k+l$, the maximal probability for $\mu=\frac12$ is

$$\frac{\left(\frac12\right)^n}{3^{\lceil n/2\rceil}\left(\frac14\right)^n+\left(\frac12\right)^n+3^{\lfloor n/2\rfloor}\left(\frac14\right)^n}=\frac1{1+2^{-n}\left(3^{\lceil n/2\rceil}+3^{\lfloor n/2\rfloor}\right)}\;.$$

The value for $n=2m+1$ is the same as for $n=2m$, so we only have to consider $n=2m$, which yields

$$\frac1{1+2^{1-2m}3^m}=\frac1{1+2\left(\frac34\right)^m}\stackrel!\gt\frac12\;,$$

and thus $1+2\left(\dfrac34\right)^m\lt2$, that is $2\left(\dfrac34\right)^m\lt1$, and thus $m\gt\dfrac{\log\frac12}{\log\frac34}\approx2.4$.

Thus you need at least $6$ tosses, and one sequence of results that would suffice is three heads, three tails (in any order), leading to a probability of $32/59\approx0.54$ for $\mu=\frac12$.

-
+1 It is indeed mildly curious that 3 heads and 4 tails gives the same posterior probability for $\mu = \frac12$ as 3 heads and 3 tails. (Even more simply, the outcome of the first toss has no impact on posterior probability for $\mu = \frac12$.) – Henry Feb 12 '13 at 23:55
@Henry: Why is that curious? If you have the same number of heads and tails so far (possibly zero), then you know that you'll get either heads or tails next, and this will tell you nothing new about the fairness of the coin, so it shouldn't change the posterior probability for $\mu=\frac12$. – joriki Feb 12 '13 at 23:57
If you have a different number of heads and tails so far, then you still know that you'll get either heads or tails next, but which will suggest something about the fairness of the coin. Perhaps in this Bayesian context I should have said mildly curious to me – Henry Feb 12 '13 at 23:59
@Henry: The "mildly" wasn't there yet when I replied :-) Yes, you always know you'll get either heads or tails, but only if you got the same number of them so far are the two situations that would arise from these two possibilities equivalent by symmetry, so you can't gain information from seeing which of them occurs. – joriki Feb 13 '13 at 0:03
I doubt if it is as small as 6 tosses. Intuitively I would expect the number of tosses needed to be quite large. Notice you have $\mu = \frac{1}{4}$ and $\mu = \frac{3}{4}$ to beat out in a "Bayesian competition". I also don't follow @joriki 's argument of "given (k+l)...", we are asked to estimate that minimum sum right? – broccoli Feb 13 '13 at 4:27