Is it true that for $Y$ and $Z$ independent, the conditional entropy $H(X\mid Y,Z)$ satisfies $$H(X\mid Y,Z) = H(X\mid Y) + H(X\mid Z)$$ where $H(X\mid Y) = \sum_{y\in\mathcal{Y}}p(y)H(X\mid Y=y)$ (similarly for $H(X\mid Z)$)?

Is this obvious? Can someone point me in the direction of a proof?

  • $\begingroup$ No, it is not true. Take the example of $Y$ and $Z$ being independent Bernoulli-1/2 bits, and $X = Y \oplus Z$, where $\oplus$ is the XOR function. It is easy to show that $H(X|Y) = H(X|Z) = 1$ but $H(X|Y,Z)$ is obviously $0$. $\endgroup$ – stochasticboy321 Nov 8 at 16:11
  • $\begingroup$ Thanks @stochasticboy321. Do you know of any identities that allow me to rewrite $H(X\mid Y,Z)$ in a different form? $\endgroup$ – jonem Nov 8 at 16:14
  • $\begingroup$ Sure, but their usefulness will depend on what you're trying to do with them, so I'm not sure if what I think of will be helpful. For example, using the chain rule of entropies, $H(X|Y,Z) = H(X,Y,Z) - H(Y,Z) = H(X,Y,Z) - H(Y) - H(Z)$ (the last equality uses independence of $Y,Z$). $\endgroup$ – stochasticboy321 Nov 8 at 16:21
  • $\begingroup$ @stochasticboy321 That's useful, thanks! $\endgroup$ – jonem Nov 8 at 16:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.