• 2 coins: $C_1$ and $C_2$
  • $p$: probability of choosing $C_1$ to flip.
  • $p_1$: probability of heads landing on $C_1$.
  • $p_2$: probability of heads landing on $C_2$.
  • All trials are Bernoulli. Which means, $P($choosing $C_2) = (1-p)$, $P($getting tail on $C_1) = (1-p_1)$, $P($getting tail on $C_2) = (1-p_2)$

I was trying to devise a formula for $p$, $p_1$ and $p_2$ using the maximum likelihood estimate technique.

My professor gave me hint to use a hidden random variable $Z$, where $Z$ is 1 when $C_1$ is chosen and 0 when $C_2$ is chosen. Then he asked me to maximize the following equation:

$$ \prod_{i=1}^N \left[p p_1^{x_i}(1-p_1)^{1-x_i}]^{z_i}[(1-p)p_2^{x_i}(1-p_2)^{1-x_i}\right]^{1-z_i}$$

Can somebody please explain me how the above equation came?


The idea is that you perform the experiment with $N$ observations, the $i^{\rm th}$ observation is an ordered pair $(z_i, x_i)$, where $z_i = 1$ if $C_1$ is chosen, and $z_i = 0$ if $C_2$ is chosen; then $x_i = 1$ if the outcome of the flip is heads, and $x_i = 0$ if tails. So your sample is a vector of ordered pairs; alternatively, you can think of it as two vectors of length $n$: $$\boldsymbol z = (z_1, \ldots, z_N), \quad \boldsymbol x = (x_1, \ldots, x_N).$$

Now, you have in essence a Bernoulli/Bernoulli hierarchical model: the choice of which $p_i$ to take depends on a second Bernoulli outcome with parameter $p$. So your likelihood will look like a composition of Bernoulli probability mass functions. For each $i$ such that $z_i = 1$, the coin $C_1$ is flipped, and the outcome of that flip is heads with probability $p_1$, and not heads with probability $1-p_1$; therefore, each such outcome contributes $$p_1^{x_i} (1-p_1)^{1-x_i}$$ to the likelihood. But this only happens when $z_i = 1$ with probability $p$; thus the subset of the sample for which $C_1$ is tossed contributes a total of $$\prod_{i=1}^N (p \cdot p_1^{x_i}(1-p_1)^{1-x_i})^{z_i}.$$ When $z_i = 0$ with probability $1-p$, $C_2$ is tossed and heads are obtained with probability $p_2$, thus this portion of the likelihood looks like $$\prod_{i=1}^N ((1-p)p_2^{x_i}(1-p_2)^{1-x_i})^{1-z_i}.$$ Put together, the total likelihood is $$\mathcal L(p, p_1, p_2 \mid \boldsymbol z, \boldsymbol x) = \prod_{i=1}^n (p \cdot p_1^{x_i} (1-p_1)^{1-x_i})^{z_i} ((1-p) p_2^{x_i}(1-p_2)^{1-x_i})^{1-z_i}.$$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.