Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I've been studying the Expectation Maximization algorithm. According to the formula shown here, what I have to do in the M step is to compute a new $\theta$ that maximizes the conditional expectation of the log function, which is $\ln P[X, z|\theta] $:

( The picture above can be acquired in page 8 of this tutorial: )

However, in the coin toss example below:

$\ln P[X, z|\theta] $ is nowhere to be found, and they don't prove how the new $\theta^{t+1}$ they got after each iteration is better than the $\theta^t$ previously acquired.

share|cite|improve this question

Note that $$\sum_z P[z|X,\theta_n] \ln P[X,z|\theta] \\=\sum_\color{red}z\bigg(\ln P[X,z|\theta]\bigg)\color{red}{P[z|X,\theta_n]} \\=E_{Z|X,\theta_n}\bigg(\ln P[X,z|\theta]\bigg)$$ Here probability distribution is $P[z|X,\theta_n]$ and summing over $z$. So, by the definition of expectation we get the desired result.

share|cite|improve this answer
I understand how the formulas work. However, what I really don't get is the way Theta is found in the coin example. They don't show or prove that the new found Theta will maximize the conditional expectation of the log function. – IcySnow May 18 '13 at 5:52
@IcySnow: I think you need to edit your question and explain your problem more briefly. I don't understand your problem still and think others also not understand otherwise they can answer your question. – Argha May 18 '13 at 6:04
I edited my question. Hopefully it makes more sense now. If there's still something wrong with it, please let me know and I will try my best to elaborate. – IcySnow May 18 '13 at 6:35

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.