I can not prove this inequality $2H(a,b,c) \leq H(a,b) + H(a,c) + H(b,c|a),\ H-entropy $. I tried do it by using chain rule and this inequality $H(X|Y) \leq H(X;Y)$ but without any success. Please help me to understand how to prove this type of inequalities. Thank you in advance.


$$H(a,b,c) = H(a)+H(b,c\mid a) \tag{1}$$ $$H(a,b) = H(a) + H(b\mid a) \tag{2}$$ $$H(a,c) = H(a) + H(c\mid a) \tag{3}$$

Hence $$2H(a,b,c) = 2 H(a) + 2H(b\mid c,a)=\\=H(a,b)+H(a,c) + H(b,c\mid a) + H(b,c\mid a)- H(b|a)-H(c\mid a) \tag{4}$$

Then we need to show that

$$ H(b,c\mid a)- H(b\mid a)-H(c\mid a) \le 0$$


$$ H(b,c\mid a) \le H(b\mid a)+H(c\mid a) \tag{5}$$

But this is true, because it's the conditioned version of

$$H(X,Y) \le H(X) + H(Y)$$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.