1
$\begingroup$

Consider two identically distributed independent random vectors $X$ and $Y$ of dimension $n$ (assume $n$ is large). Both $X$ and $Y$ have only positive integer entries in some finite range. I am trying to understand what conditions are sufficient for $H(X) + H(Y) \approx H(X+Y)$?

Is it possible to say what properties of the random distributions are sufficient for this to be true?

$\endgroup$
1
$\begingroup$

You can say that $$ H(X+Y)=H(X+Y | X) + H(X) - H(X | X+Y)=H(Y)+H(X)-H(X|X+Y), $$ so $H(X+Y) \approx H(X) + H(Y)$ iff $H(X|X+Y)\approx 0$, i.e., if you can essentially determine what $X$ must be when given $X+Y$. For instance, if the entries of $Y$ are always even, and the entries of $X$ can only be zero or one, then $H(X+Y)=H(X)+H(Y)$ exactly.

$\endgroup$
  • $\begingroup$ This is very helpful but in my case $X$ and $Y$ are i.i.d. Is there a simple example that would work? $\endgroup$ – dorothy Dec 11 '15 at 21:36
  • $\begingroup$ Entries of $X$ and $Y$ can be drawn from $\{1,2,4,8,\ldots, 2^n\}$. $\endgroup$ – mjqxxxx Dec 12 '15 at 1:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.