Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

How do we derive the following: $$ -\sum_{x} \sum_{y} p(x,y) \left( \log_{2} p(x) + \log_{2} p(y) - \log_{2} p(x,y) \right)$$ $$= -\sum_{y} p(y) \log_{2} p(y)+\sum_{x} p(x) \sum_{y} p(y|x) \log_{2} p(y|x)$$

The first equation simplified is: $$-\sum_{x} \sum_{y} p(x,y) \log_{2} p(x)-\sum_{x}\sum_{y} p(x,y) \log_{2} p(x,y)+\sum_{x}\sum_{y} p(x,y) \log_{2} p(x,y)$$

Do we just use the fact that $p(x,y) = p(y)p(x|y) = p(x)p(y|x)$ and substitute in?

share|improve this question
1  
What do you get after doing the substitution? –  Alex R. Feb 1 '13 at 0:15
add comment

1 Answer 1

This is just a guess: We know that $$-\sum_{x} \sum_{y} p(x,y) \log_{2} p(y) = -\sum_{y} \sum_{x} p(x,y) \log_{2} p(y)$$

$$= -\sum_{y} p(y) \log_{2} p(y)$$

Also $$\sum_{x} \sum_{y} p(x,y) \log_{2}p(x,y)- \sum_{x} \sum_{y} p(x,y) \log_{2} p(x)$$ $$ = \sum_{x} \sum_{y} p(x,y) \log_{2} \frac{p(x,y)}{p(x)}$$

$$ = \sum_{x} \sum_{y} p(y|x) p(x) \log_{2} p(y|x)$$

$$= \sum_{x} p(x) \sum_{y} p(y|x) \log_{2} p(y|x)$$

Thus $$ -\sum_{x} \sum_{y} p(x,y) \left( \log_{2} p(x) + \log_{2} p(y) - \log_{2} p(x,y) \right)$$ $$= -\sum_{y} p(y) \log_{2} p(y)+\sum_{x} p(x) \sum_{y} p(y|x) \log_{2} p(y|x)$$

share|improve this answer
    
The first thing that We know in your post seems wrong. –  Did Feb 1 '13 at 16:35
    
@Did: What is wrong about it? –  proton Feb 1 '13 at 16:36
    
Try some concrete cases. You seem to have confused $\log p(x,y)$ with $\log p(y)$. –  Did Feb 1 '13 at 16:45
    
@Did: I was using the fact that $\sum_{x} p(x,y) = p(y)$. Then $\sum_{x} p(x,y) \log_{2} p(x,y) = p(y) \log_{2} p(y)$. –  proton Feb 1 '13 at 16:51
    
No. As I already indicated, this would be true if one had $\log p(y)$ instead of $\log p(x,y)$ in the summation over $(x,y)$. –  Did Feb 1 '13 at 17:03
show 7 more comments

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.