2
$\begingroup$

[Editing question per Leon's suggestions - thanks for these!]

Could someone walk me through a solution to Ex 2.8?

2.7: Bill tosses a bent coin $N$ times, obtaining a sequence of heads and tails. We assume that the coin has a probability $f_H$ of coming up heads; we do not know $f_H$. If $n_H$ heads have occurred in $N$ tosses, what is the probability distribution of $f_H$? ...What is the probability that the $N+1$th outcome will be a head, given $n_H$ heads in $N$ tosses?

2.8: Assuming a uniform prior on $f_H$, $P(f_H)=1$, solve the problem posed in 2.7. Sketch the distribution of fH and compute the probability that the $N+1$th outcome will be a head, for (A) $N=3$ and $n_H$=0; (B) $N=3$ and $n_H=2$; (C) $N=10$ and $n_H=3$; (D) $N=300$ and $n_H=29$.

{tip about the beta integral}

Where I am stuck is with the switch to continuous probabilities, and using integrals rather than sums. Had no problem with 2.4; 2.5 took some doing but was fine. The example in 2.6 made sense walking through it.

In working on 2.8, I can write down that posterior = (likelihood x prior) / evidence, and know that I am trying to solve for posterior (to find the distribution of $f_H$). So my equation will look something like

$$P(f_H |\,n_H, N) = {P(n_H|\,f_H, N) P(f_H) \over P(n_H|\,N)}$$.

The left hand side of the numerator should just be the binomial probability

${N \choose n_H}$ $f_H^{n_H}$ $(1-f_H)^{N-n_H}$ Based on the statement in the question, I assume that $P(f_H) = 1$ and ignore it.

The denominator is the marginal probability of $n_H$. I believe this should be an integral - something like $\int_0^1 P(f_H) P(n_H |\,f_H, N) df_H$. But I am not sure that this is correct, and am not sure how to solve it, even with the hints.

I did notice that 2.7 is an example and the additional assumptions - but need help here too.

Thank you in advance

[Not technically homework as I'm not doing this as part of a course, but it's close enough to tag it]

$\endgroup$
4
$\begingroup$

The question is essentially solved in §3.2. Use the equation: $$posterior = \frac{likelihood \times \ prior}{evidence}$$ and work it through.

Assume a uniform prior - $P(f_H) = 1$.

The evidence $P(n_H| \ N)$ is the normalizing constant - $\int_0^1 df_H \ f_H^{n_H} {(1-f_H)}^{N-n_H}$.

Based on the hint about the beta integral, $\int_0^1 df_H \ f_H^{n_H} {(1-f_H)}^{N-n_H} = \frac{n_H! \ (N-n_H)!}{(N+1)!}$.

The likelihood is $P(n_H| \ f_H,N) = f_H^{n_H} {(1-f_H)}^{N-n_H}$

So the posterior probability is just the ratio of the likelihood over the evidence.

$$P(f_H | \ n_H, N) = \frac{f_H^{n_H} {(1-f_H)}^{N-n_H}}{\frac{n_H! \ (N-n_H)!}{(N+1)!}}$$

To find the probability of the $N+1^{th}$ toss, integrate over $f_H$. By the sum rule, $$P(h \ |\ n_H, N) = \int df_H P(h | \ f_H) \ P(f_H | \ n_H, N) $$

$P(h| \ f_H) = f_H$, so we have $\int_0^1 df_H\frac{f_H^{n_H+1} {(1-f_H)}^{N-n_H}}{\frac{n_H! \ (N-n_H)!}{(N+1)!}}$

Using the beta integral again, this becomes $$\frac{(n_H+1)! (N-n_H)!}{(N+2)!} \ x \ \frac{(N+1)!}{n_H! (N-n_H)!} = \frac {n_H +1}{N+2}$$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.