My question is really a request for a clarification, and pertains to whether or not there are questions for which it is necessary to use EM or if it's more of a convenience.

Let's presume we have n normal distributions with unknown variance and unknown means. The data we obtain are a series of m trials of the form: "sample a was drawn from distribution A and sample b was drawn from distribution B and b was found to be greater than a."

We wish to take these trials and from them and estimate the means of the distributions (it would be nice, but not necessary, to obtain the variances as well).

Based on a previous question and my research, it seems this can be estimated with:

$${\mu _1},{\mu _2}, \ldots ,{\mu _m} \arg \max= \sum\limits_{\left( {i,j} \right) \in S} {C_i,_j\log \Phi \left(\mu_i-\mu_j \right)} $$

or (possibly)

$${\mu _1},{\mu _2}, \ldots ,{\mu _m} \arg\max = \sum\limits_{\left( {i,j} \right) \in S} {C_i,_j\log \Phi \left( \frac{\mu_i-\mu_j}{\sqrt{\sigma_i^2+\sigma_j^2}} \right)} $$

Now, I can envision doing this two ways: performing maximum likelihood estimation for all parameters, i.e., both means and variances, or, we could perform expectation maximization, treating the variance of each distribution as a 'missing' parameter, which I understand is one case in which EM is useful.

So, I have several questions:

(1) Is it the case that all problems solved with EM can also be solved with other methods, like maximum likelihood?

(2) If (1) is true, does EM simply reduce the complexity of the calculation?

(3) Is it invalid to use EM to estimate parameters if one of the hidden parameters is also of interest?

(4) Hypothetically, let's say that we just want to estimate the means. Would it be invalid to declare half the the means as 'missing parameters', and then alternate estimating one half then the other via EM?


To answer your first and second question: I think that all problems solved with EM can also be solved with maximum likelihood. EM is an optimization method. What makes EM attractive is that for certain problems it simplifies the computation and is easier to implement.

I would like to know of instances where EM is indispensable. I joined this forum to ask exactly the same question!


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.