4 votes

Joint entropy of a random variable with itself

We can directly compute the case of when they are identical: $$H(A, B) = \sum_{a, b} p(a, b) \log p(a,b) = \sum_{a = b} p(a, b) \log p(a,b)$$ where in the second equality we applied the assumption ...
user avatar
  • 1,764
4 votes

Joint entropy of a random variable with itself

But what is $H(AB)$ if A and B are not independent, for example, if they are identical? dmh has posted the calculation which demonstrates that two identical distributions have the same entropy as ...
user avatar
  • 8,210

Only top scored, non community-wiki answers of a minimum length are eligible