-1
$\begingroup$

I've been studying the Expectation Maximization algorithm. According to the formula shown here, what I have to do in the M step is to compute a new $\theta$ that maximizes the conditional expectation of the log function, which is $\ln P[X, z|\theta] $: http://i86.photobucket.com/albums/k118/ProtoMan_03/expectation_zps2689ab59.jpg

( The picture above can be acquired in page 8 of this tutorial: http://www.seanborman.com/publications/EM_algorithm.pdf )

However, in the coin toss example below: http://www.nature.com/nbt/journal/v26/n8/full/nbt1406.html?pagewanted=all

$\ln P[X, z|\theta] $ is nowhere to be found, and they don't prove how the new $\theta^{t+1}$ they got after each iteration is better than the $\theta^t$ previously acquired.

|cite|improve this question
$\endgroup$
1
$\begingroup$

Note that $$\sum_z P[z|X,\theta_n] \ln P[X,z|\theta] \\=\sum_\color{red}z\bigg(\ln P[X,z|\theta]\bigg)\color{red}{P[z|X,\theta_n]} \\=E_{Z|X,\theta_n}\bigg(\ln P[X,z|\theta]\bigg)$$ Here probability distribution is $P[z|X,\theta_n]$ and summing over $z$. So, by the definition of expectation we get the desired result.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ I understand how the formulas work. However, what I really don't get is the way Theta is found in the coin example. They don't show or prove that the new found Theta will maximize the conditional expectation of the log function. $\endgroup$ – IcySnow May 18 '13 at 5:52
  • 1
    $\begingroup$ @IcySnow: I think you need to edit your question and explain your problem more briefly. I don't understand your problem still and think others also not understand otherwise they can answer your question. $\endgroup$ – Argha May 18 '13 at 6:04
  • $\begingroup$ I edited my question. Hopefully it makes more sense now. If there's still something wrong with it, please let me know and I will try my best to elaborate. $\endgroup$ – IcySnow May 18 '13 at 6:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.