If I have joint entropy = 10 bits and mutual information = 2 bits, could we say our two distributions have only 0.4% of useful communication overall?

Or to say it differently: 99,6% is complete noise.

4/1024 = 0.004 = 0.4%


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.