Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I proved that

I(X,Y) <= 1 - H(p)

to the following way: enter image description here

How can I prove if I start in that way I(X,Y) = H(X) - H(X|Y), I know we can deduct I(X,Y) = H(X) - H(X|Y), H(X,Y) = H(X,Y) - H(Y) and H(X,Y) = H(Y|X) + H(X) => I(X,Y) = H(X) - H(Y|X) - H(X) + H(Y) but this is the first equality.

If it is impossible why, can i prove it a logic way?

share|improve this question
You would require the Bayes rule to compute the probabilities. Since this problem is small, it can be done. But in general, the identity $I(X;Y) = H(X)-H(X|Y) = H(Y)-H(Y|X)=I(Y;X)$ is very useful in one direction and painful in the other. Try other channels like the multiplicative channel and noisy typewriter. Refer "Elements of Information Theory" by Cover and Thomas for more details. –  Gautam Shenoy Nov 13 '12 at 12:32
I already download the book, but only I find the first in the book. Thanks for your answer. –  Tatar Elemér Nov 13 '12 at 12:50
Look at the later chapters. Not just the first two. Look at the index as well. It may be in the problem section. –  Gautam Shenoy Nov 14 '12 at 5:10

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.