I am having problems finding a well thought out complete explanation of expectation maximization. Does anyone have a best source for someone completely new to this stuff?

  • 1
    $\begingroup$ Do you mean something other than the original paper, discussions, and rejoinder? $\endgroup$
    – cardinal
    Mar 25, 2012 at 1:17
  • $\begingroup$ If you just want it for applications I think the wikipedia page is coherent. @cardinal If you want something rigorous, IIRC the original paper (Dempster, Laird, and Rubin) failed to rigorously prove that the EM algorithm converges to a local max. $\endgroup$
    – guy
    Mar 25, 2012 at 1:54
  • $\begingroup$ @guy: Yes, but that is covered by Wu (1983) which is not hard to find either. :) $\endgroup$
    – cardinal
    Mar 25, 2012 at 2:16
  • $\begingroup$ But are any of these good for someone who is a newbie at mathematics and statistics? $\endgroup$
    – user782220
    Mar 25, 2012 at 2:57
  • $\begingroup$ I am currently working on implementing such algorithm and have a convergence issue... My code converges to a solution after only the second step but it converges to a trivial solution, i.e. all-zero values in the solution. And I clearly don't expect that to be a solution... Has anyone ever encountered such convergence issue? I overlooked at my implementation and can't see why I get to that situation. Perhaps I don't fully understand what is going on in that iterative process... Any hint will be welcome. $\endgroup$
    – EricP
    Aug 30, 2012 at 20:18

1 Answer 1


Check out the following tutorials:

T. K. Moon, "The expectation-maximization algorithm", IEEE Signal Processing Magazine, vol. 13, no. 6, pp. 47-60, 1996.

J. A. Bilmes, "A gentle tutorial of the EM algorithm and its application to parameter estimation for Gaussian mixture and hidden Markov models".

Of course, you can refer to the original paper by Dempster et al. But it might be slightly hard for a first read.

Another reference is the Pattern Recognition and Machine Learning book by C. Bishop. It has a nice (and intuitive) explanation for EM.


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .