I am having problems finding a well thought out complete explanation of expectation maximization. Does anyone have a best source for someone completely new to this stuff?

  • 1
    $\begingroup$ Do you mean something other than the original paper, discussions, and rejoinder? $\endgroup$ – cardinal Mar 25 '12 at 1:17
  • $\begingroup$ If you just want it for applications I think the wikipedia page is coherent. @cardinal If you want something rigorous, IIRC the original paper (Dempster, Laird, and Rubin) failed to rigorously prove that the EM algorithm converges to a local max. $\endgroup$ – guy Mar 25 '12 at 1:54
  • $\begingroup$ @guy: Yes, but that is covered by Wu (1983) which is not hard to find either. :) $\endgroup$ – cardinal Mar 25 '12 at 2:16
  • $\begingroup$ But are any of these good for someone who is a newbie at mathematics and statistics? $\endgroup$ – user782220 Mar 25 '12 at 2:57
  • $\begingroup$ I am currently working on implementing such algorithm and have a convergence issue... My code converges to a solution after only the second step but it converges to a trivial solution, i.e. all-zero values in the solution. And I clearly don't expect that to be a solution... Has anyone ever encountered such convergence issue? I overlooked at my implementation and can't see why I get to that situation. Perhaps I don't fully understand what is going on in that iterative process... Any hint will be welcome. $\endgroup$ – EricP Aug 30 '12 at 20:18

Check out the following tutorials:

T. K. Moon, "The expectation-maximization algorithm", IEEE Signal Processing Magazine, vol. 13, no. 6, pp. 47-60, 1996.

J. A. Bilmes, "A gentle tutorial of the EM algorithm and its application to parameter estimation for Gaussian mixture and hidden Markov models".

Of course, you can refer to the original paper by Dempster et al. But it might be slightly hard for a first read.

Another reference is the Pattern Recognition and Machine Learning book by C. Bishop. It has a nice (and intuitive) explanation for EM.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.