Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I am having problems finding a well thought out complete explanation of expectation maximization. Does anyone have a best source for someone completely new to this stuff?

share|improve this question
1  
Do you mean something other than the original paper, discussions, and rejoinder? –  cardinal Mar 25 '12 at 1:17
    
If you just want it for applications I think the wikipedia page is coherent. @cardinal If you want something rigorous, IIRC the original paper (Dempster, Laird, and Rubin) failed to rigorously prove that the EM algorithm converges to a local max. –  guy Mar 25 '12 at 1:54
    
@guy: Yes, but that is covered by Wu (1983) which is not hard to find either. :) –  cardinal Mar 25 '12 at 2:16
    
But are any of these good for someone who is a newbie at mathematics and statistics? –  user782220 Mar 25 '12 at 2:57
    
I am currently working on implementing such algorithm and have a convergence issue... My code converges to a solution after only the second step but it converges to a trivial solution, i.e. all-zero values in the solution. And I clearly don't expect that to be a solution... Has anyone ever encountered such convergence issue? I overlooked at my implementation and can't see why I get to that situation. Perhaps I don't fully understand what is going on in that iterative process... Any hint will be welcome. –  EricP Aug 30 '12 at 20:18
show 1 more comment

1 Answer

Check out the following tutorials:

T. K. Moon, "The expectation-maximization algorithm", IEEE Signal Processing Magazine, vol. 13, no. 6, pp. 47-60, 1996.

J. A. Bilmes, "A gentle tutorial of the EM algorithm and its application to parameter estimation for Gaussian mixture and hidden Markov models".

Of course, you can refer to the original paper by Dempster et al. But it might be slightly hard for a first read.

Another reference is the Pattern Recognition and Machine Learning book by C. Bishop. It has a nice (and intuitive) explanation for EM.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.