This question is from Nielsen & Chuang "Quantum Computation & Quantum Information Theory", Chapter 11, Exercise 11.7.

Find an expression for the conditional entropy H(Y|X) as a relative entropy between two probability distributions.

I have been unsuccessful in my attempts to prove this result so far, and any help would be appreciated. I'm reading thorough this book by myself, and have been stuck on this problem for quite some time now. I have not managed to find a solution on the internet for this, even after substantial amount of search.

  • $\begingroup$ @JohnPolcari Not really... (see e.g. this) $\endgroup$ – Clement C. Jul 21 '18 at 21:02
  • $\begingroup$ After some further thought, I believe that there is probably no way to express conditional entropy as a relative entropy, and that the question was most likely looking for the "not quite" result pointed out by @Clement. $\endgroup$ – John Polcari Jul 21 '18 at 22:41
  • $\begingroup$ @JohnPolcari But then, how would you use that to show the conditional entropy is non-negative? (This is actually what the exercise in question asked, as a conclusion.) That "not quite result" expresses the conditional entropy as the opposite of a "not really a relative divergence"... $\endgroup$ – Clement C. Jul 21 '18 at 22:45
  • $\begingroup$ @Clement, I could see expressing it "in terms of" but not "as", which to me connotes no other terms are allowed. Do you interpret it as the looser meaning?: $\endgroup$ – John Polcari Jul 21 '18 at 23:07
  • $\begingroup$ @JohnPolcari it's mostly that I don't see how to use properties of the relative entropy (it's non-negative) to derive properties of this "negated not-really-a-relative-entropy" quantity. $\endgroup$ – Clement C. Jul 21 '18 at 23:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.