Expected Value for Heads for Unknown Weighted Coin Given Head First Flip

Question

This is a combinatorics problem, and I think it involves expected values and conditional probability, but I don't know how to use them:

"A bag contains an infinite number of coins whose probabilities of heads on any given flip are uniformly and continuously distributed between 0 and 1. A coin is drawn at random from this bag. Given that the first flip is a head, determine the probability that the next flip is also a head."

The answer is 2/3, but could someone please explain why?

Answer 1

Using the definition of conditional probability, $$P(H_2 \mid H_1) = \frac{P(H_1 \cap H_2)}{P(H_1)}$$ where $H_1, H_2$ are events of getting heads on the first flip and the second flip, respectively. To get the marginal probability, $$P(H_1) = \int_{p'=0}^{1}P(H_1 \mid p = p') dp'$$ where $(p = p')$ is the event of choosing a coin with probability $p'$ of flipping heads. Since $P(H_1 \mid p = p') = p'$: $$P(H_1) = \int_{p'=0}^{1} p' dp' = \frac{1}{2}$$ Similarly, $$P(H_1 \cap H_2) = \int_{p'=0}^{1} P(H_1 \cap H_2 \mid p = p') dp' = \int_{p' = 0}^{1} p'^2 dp' = \frac{1}{3}$$

Therefore,

$$P(H_2 \mid H_1) = \frac{2}{3}$$

Answer 2

See the rule of succession which gives $P(X_2=H|X_1=H)=\frac{s+1}{n+2}=\frac{2}{3}$ since $s=1$ is the number of heads so far observed and $n=1$ is the number of trials.

Answer 3

Let random variable $P_H$ be the bias towards heads of the coin you picked.

$$\begin{align}\mathsf P(H_2\mid H_1) =&~ \dfrac{\mathsf P(H_1\cap H_2)}{\mathsf P(H_1)} & \text{by definition} \\[1ex] =&~ \dfrac{\mathsf E(\mathsf P(H_1\cap H_2\mid P_H))}{\mathsf E(\mathsf P(H_1\mid P_H))} & \text{Law of Total Probability} \\[1ex] =&~ \dfrac{\mathsf E({P_H}^2)}{\mathsf E(P_H)} \\[1ex] =&~ \dfrac{\int_0^1 p^2\operatorname d p}{\int_0^1 p\operatorname d p} & P_H\sim\mathcal U[0;1]\\[1ex] =&~ \dfrac{\tfrac 13(1^3-0^3)}{\tfrac 1 2(1^2-0^2)}\\[1ex] =&~\dfrac 2 3\end{align}$$