1
$\begingroup$

I am trying to study EM algorithm and Maximum Likelihood Estimation. Somehow, they both sound the same to me but can't really say the difference. Maybe I don't really understand any of them. I have just started.

Can somebody tell me what they do and the relationship or difference between them?

$\endgroup$
  • 3
    $\begingroup$ MLE gives you the objective to maximize. EM is one method for finding this maximum. In other words, EM finds (or attempts to find) the MLE iteratively. There are other methods, e.g. pick any optimization method. $\endgroup$ – Chester Aug 9 '15 at 14:55
  • 1
    $\begingroup$ I believe that's very broad question. One possible difference: if you want to estimate parameters $\alpha_i$, $\mu_i$ and $\sigma_i$ of Gaussian mixture $\sum_i \alpha_i N(\mu_i, \sigma_i)$, you cannot use MLE and you must use EM-algorithm instead and find parameters iteratively. On the other hand, when $\alpha_i$'s are known, you can use MLE. It is like directly solving an equation if possible (MLE), or using some iterative numeric method otherwise. $\endgroup$ – Antoine Aug 9 '15 at 14:57
  • $\begingroup$ When can I use em and when I cannot use it? How can I distinguish? $\endgroup$ – user122358 Aug 9 '15 at 15:13
  • 1
    $\begingroup$ EM is computationally more expensive, so use MLE if possible. Try to use MLE In my upper example when $\alpha_i$'s are unknown (really - try it, that helped me, when I came across EM algorithm). Then, you will understand in which cases you cannot use MLE. (When some of your parameters (in our case $\alpha_i$'s) are unobservable.) $\endgroup$ – Antoine Aug 9 '15 at 16:04
  • 1
    $\begingroup$ @user122358 Assume you have three types $Y_i$ of people: children, grown up men and grown up women and you measure their height $X$. Suppose it's distribution is given as $X\mid Y_i \sim N(\mu_i, \sigma_i)$. $\alpha_i$ is probability that height $X$ belongs to a person of type $Y_i$. $\endgroup$ – Antoine Aug 13 '15 at 12:22
1
$\begingroup$

As per my understanding

EM

Assumption: The given data is coming/originated from multiple classes/distribution

Input: Data points

Output: Each data point classified to a distribution and the parameters of the distribution.

MLE

Assumption: The distribution from which the data points originate

Input: Data points

Output: Parameters of the distribution which maximizes the probability that the data points have originated from that particular distribution.

If this is correct, then EM might be doing MLE as an intermediate step.

Please correct me if I am wrong.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.