Problem: Consider a simple coin-flipping experiment in which we are given a pair of coins A and B of unknown biases, $\theta_{A}$ and $\theta_{B}$ respectively (that is, on any given flip, coin A will land on heads with probability $\theta_{A}$ and tails with probability $1-\theta_{A}$ and similar for coin B).

Repeating the following procedure five times: randomly choose one of the two coins, and perform 10 independent coin tosses with the selected coin. The probability of selecting coin A and coin B for each set is equal and this selection is made once per set of 10 tosses.

Suppose that we keep track of two vectors $x=(x_{1},x_{2},\ldots,x_{5})$ and $z=(z_{1},z_{2},\ldots,z_{5})$, where $x_{i}\in\{0,1,\ldots,10\}$ is the number of heads observed during the $i^{th}$ set of tosses, and $z_{i}\in\{A,B\}$ is the identity of the coin used during the $i^{th}$ set of tosses. Then the outcomes of 5 sets, can be represented by a vector $y=(y_{1},y_{2},y_{3},y_{4},y_{5})$. Each element in vector y, is denoted by $y_{i}=(x_{i},z_{i})$.

For example, $y_{3}=(x_{3}=8,z_{3}=A)$ means that there are 8 heads among 10 tosses for the 3rd set given coin A is chosen.

Let $f(y|\theta)$ denote the probability density function (PDF) that specifies the probability of observing data vector $y=(y_{1},y_{2},\ldots,y_{i},\ldots,y_{5})$ given the parameter $\theta=(\theta_{A},\theta_{B})$. If individual observation, $y_{i}$'s are statistically independent of one another, then according to the theory of probability, the PDF for the observation data $y=(y_{1},y_{2},\ldots,y_{i},\ldots,y_{5})$ given the parameter vector $\theta=(\theta_{A},\theta_{B})$ can be expressed as a multiplication of PDFs for individual observations

\begin{equation} \label{1} f(y=(y_{1},y_{2},\ldots,y_{i},\ldots,y_{5})|\theta)=f_{1}(y_{1}|\theta)f_{2}(y_{2}|\theta)\ldots f_{5}(y_{5}|\theta) \end{equation}

In a simplest case, there are only one observation and one parameter, that is, $y$ is a single element variable, and $\theta$ is a single element constant. Suppose that the single element independent variable y represents the number of heads observed in a single set of 10 Bernoulli trials(e.g. tossing a coin 10 times) and that the probability of a head on any one trial, represented by the single element parameter $\theta$; is 0.8. The PDF in this case is given by \begin{equation} f(y|n=10,\theta=0.8)=\frac{10!}{y!(10-y)!}(0.8)^{y}(0.2)^{10-y} \end{equation} where (y=0,1,...,10).

In the case I described at the beginning, there are 5 observations and two parameters, that is, the independent variable of y is a 5-element variable here $y=(y_{1},y_{2},\ldots,y_{i},\ldots,y_{5})$ and the parameter $\theta$ is a two element vector $\theta=(\theta_{A},\theta_{B})$ given that $\theta_{A}=0.8$, $\theta_{B}=0.45$.

Q: what is equation of PDF of $f(y=(y_{1},y_{2},\ldots,y_{i},\ldots,y_{5})| \theta=(\theta_{A}=0.8,\theta_{B}=0.45))$ here?

The reason I want to ask this question is to find out the maximum likelihood function since MLE is a inverse problem here.



There are three problems in your question that I think need to be addressed before it can be meaningfully answered.

  • The specification $n=50$ appears out of nowhere at the end; you had neither introduced a variable $n$ for the number of tosses, nor indicated that it should be considered as a parameter and not fixed.

  • You don't specify the probabilities for coins $A$ and $B$ to be used for any of the $5$ sets of tosses. Perhaps you're assuming that they're selected with equal probabilities as in the tutorial you'd asked about in an earlier question.

  • You introduce $x$ and $z$ as vectors of five variables each recording the number of heads and the coin used, respectively, in the $i$-th set. Then you write that each $y_i=(x,z)$. That doesn't make any sense. There's a free index $i$ on the left-hand side but not on the right-hand side; but even if we drop the index and assume that you meant $y=(x,z)$, it still doesn't make any sense, since there's a vector of $50$ individual tosses on the left and a vector of aggregated data containing a total of $10$ entries on the right.

[Edit in response to the edit in the question]

The question is still unclear on all three points.

  • You've now introduced $n=50$ before using it, but it's still not clear what it's doing in the condition part of a conditional probability. This would seem to indicate that you're considering other possible values of $n$ and the value of $n$ is determined randomly; that seems to be inconsistent with the rest of your description, which assumed $n$ fixed all along.
  • I only understand "each coin is selected with equal probability" by consulting the other questions and the tutorial; to make the question self-contained, you'd need to specify that this choice is made once per set of $10$ tosses.
  • The relationship between $y$ and $(x,z)$ isn't any clearer than before. $f(x,z|\theta)$ cannot be the PDF for $y$ because $x$ and $z$ contain less information than $y$; the results for sets of $10$ tosses each are aggregated in $x$, whereas they aren't aggregated in $y$.

[Edit in response to the second edit:]

It's now slightly less difficult to guess what you're trying to get at, but your notation still needs considerable work. You don't explain the meaning of the additional indices $j$ and $k$, and I get the impression that what you in fact mean is $y_i=(x_i,z_i)$. (Also note that the $y_i$ are now defined quite differently from the version in the original question; it would help to either delete the old definition or introduce a new symbol for this new quantity instead.)

You still haven't addresses what the $n=10$ is doing in the condition part of a conditional probability – nothing else in the entire post suggests that this is an unknown parameter or a random variable.

Also, $50!$ in the numerator of your function $f$ should be $10$, and you've swapped the probabilities $\theta$ and $1-\theta$ of success and failure.

However, since I think I can now guess what you're trying to ask, here's an answer to that:


  • $\begingroup$ @user19262: I've responded to your second edit. $\endgroup$ – joriki Nov 17 '11 at 13:09
  • $\begingroup$ Thanks for you answer. @joriki: Taking your advice, I have edited the post (redefine the notations that confuse you, and delete the old post). Can you take a look at this updated post and see it is easy to read now. I can edit it again if it is still confusing. Can you also briefly explain how the PDF equation you gave to me is derived here? Thanks. I could understand the product part which is product of 5 Bernoulli distribution, but I could not figure out that (1/(2^5)) at front. $\endgroup$ – xuesong Nov 19 '11 at 14:14
  • $\begingroup$ The (1/2^5) comes from the fact that z is a random vector of length 5. Each element of the vector represents A or B and thus has 2 possible values. Thus the probability of any 1 realization of the random vector z (e.g. Z={H,H,T,H,T} or Z = {T,T,H,T,T}) has probability 1/(2^5). It might be useful to realize if x and z are independent p(x,z)=p(x)p(z). $\endgroup$ – user3731622 Sep 7 '16 at 21:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.