How to relate the calculation of expected data Problem: Consider a simple coin-flipping experiment in which we are given a pair of coins A and B of unknown biases, $\theta_{A}$ and $\theta_{B}$ respectively (that is, on any given flip, coin A will land on heads with probability $\theta_{A}$ and tails with probability $1-\theta_{A}$ and similar for coin B).

Repeating the following procedure five times: randomly choose one of the two coins, and perform 10 independent coin tosses with the selected coin. The probability of selecting coin A and coin B for each set is equal and this selection is made once per set of 10 tosses. Results are show in the example 1 a below.

Example a:

$ \begin{gather} \text{set 1: coin B : 5H 5T } \\ \text{set 2: coin A : 9H 1T }\\ \text{set 3: coin A : 8H 2T }\\ \text{set 4: coin B : 4H 6T }\\ \text{set 5: coin A : 7H 3T }\\ \end{gather} $

Suppose that we keep track of two vectors $x=(x_{1},x_{2},\ldots,x_{5})$ and $z=(z_{1},z_{2},\ldots,z_{5})$, where $x_{i}\in\{0,1,\ldots,10\}$ is the number of heads observed during the $i^{th}$ set of tosses, and $z_{i}\in\{A,B\}$ is the identity of the coin used during the $i^{th}$ set of tosses. Then the outcomes of 5 sets, can be represented by a vector $y=(y_{1},y_{2},y_{3},y_{4},y_{5})$. Each element in vector y, is denoted by $y_{i}=(x_{i},z_{i})$.

Let $f(y|\theta)$ denote the probability density function (PDF) that specifies the probability of observing data vector $y=(y_{1},y_{2},\ldots,y_{i},\ldots,y_{5})$ given the parameter $\theta=(\theta_{A},\theta_{B})$. If individual observation, $y_{i}$'s are statistically independent of one another, then according to the theory of probability, the PDF for the observation data $y=(y_{1},y_{2},\ldots,y_{i},\ldots,y_{5})$ given the parameter vector $\theta=(\theta_{A},\theta_{B})$ can be expressed as a multiplication of PDFs for individual observations

\begin{equation} %\label{1} f(y=(y_{1},y_{2},\ldots, y_{i}, \ldots,y_{5})mid\theta)=f_{1}(y_{1}\mid\theta)f_{2}(y_{2}\mid\theta)\ldots f_{5}(y_{5}\mid\theta) \end{equation}

What is the likelihood function representation here relating to this example?


  • $\begingroup$ Just to be crystal-clear: You're saying in each set we know which coin we've got. If we didn't, we'd get a different likelihood function. And when you say we randomly choose a coin, does that mean they have equal chances of being chosen? (That won't actually affect the likelihood function, beyond a constant multiple, if we know which coin we've got in each set. But it will if we don't know.) $\endgroup$ – Michael Hardy Nov 29 '11 at 6:55
  • $\begingroup$ Yes, we know which coin which have got for tossing during the set. And each coin is randomly chosen with equal chance. $\endgroup$ – xuesong Nov 29 '11 at 8:51

The probability of the outcome you've written as $\text{set 4: coin B : 4H 6T }$, given that you got that particular coin on that occasion, is $\text{constant}\cdot\theta_B^4(1-\theta_B)^6$, where in this case "constant" means not depending on the two parameters $\theta_A$ and $\theta_B$. You need to do the same thing with the other four "sets" and the multiply, getting $L(\theta_A,\theta_B)=(\mathrm{constant})\cdot(\cdots\cdots)( \theta_B^4(1-\theta_B)^6)( \cdots\cdots)$ where $(\cdots\cdots)$ represents the other four factors you need to multiply by.

For most (maybe all?) purposes the value of the "constant" is irrelevant, beyond the fact that it's positive.

  • $\begingroup$ In fact, since $x_i$ is the number of heads on the $i$-th trial and $z_i \in \{A,B\}$ tells us which coin was used on the $i$-th trial, it is possible to state more succinctly that the likelihood is proportional to $$\prod_{i=1}^5\theta_{z_i}^{x_i}(1-\theta_{z_i})^{10-x_i}$$ $\endgroup$ – Dilip Sarwate Nov 29 '11 at 18:40
  • $\begingroup$ Thanks Michael and Dilip for your answer. but I notice that the binomial coefficients are not mentioned in neither of your formula. Is $L(\theta_{z_i})=\prod_{i=1}^5\binom{10}{x_i}\theta_{z_i}^{x_i}(1-\theta_{z_i})^{10-x_i}\;$.correct or binomial coefficients are not necessary here ? $\endgroup$ – xuesong Nov 29 '11 at 23:01
  • $\begingroup$ @xuesong : The likelihood $L(\theta)$ is a function of $\theta$ and the binomial coefficient does not depend on $\theta$. Therefore its value is not relevant to the problem, beyond the fact that it is positive. $\endgroup$ – Michael Hardy May 20 '13 at 1:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.