How can be proven that for random variables $A$ and $B$, and $C = A + B$,

$$H(C\mid A) = H(B\mid A).$$

Also, would it be possible to determine if $H(C)$ would be greater than $H(A)$?

  • $\begingroup$ Differential or Shannon entropy, or some other type of entropy? $\endgroup$ – cantorhead Feb 3 '18 at 19:52
  • $\begingroup$ Seems roughly equivalent to showing $H(A,A+B)=H(A,B)$. $\endgroup$ – cantorhead Feb 3 '18 at 19:53
  • $\begingroup$ Is the fact that the operation is addition important? Should this statement also hold: $H(A,AB)=H(A,B)$, under suitable conditions? Or could it more generally be $H(A,f(A,B))=H(A,B)$? $\endgroup$ – cantorhead Feb 3 '18 at 19:54

I guess $A$ and $B$ are discrete random variables. Since $C$ is a function of $A$ and $B$ one has $H(C \mid A) \leq H(A,B \mid A)$ (because this is even true that $H(C \mid A=a) \leq H(A,B \mid A=a)$ for every $a$). Moreover $H(A,B \mid A) = H(B \mid A)$ because it is easy to check that $H(A,B \mid A=a) = H(B \mid A=a)$ for every $a$. Finally $H(C \mid A) \leq H(B \mid A)$ and one similarly gets the converse inequality.

For your second question: no, take $B=-A$ then $H(C)=0$. I have not tried to check whether this is true under the independence assumption of $A$ and $B$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.