4
votes
Joint entropy of a random variable with itself
We can directly compute the case of when they are identical:
$$H(A, B) = \sum_{a, b} p(a, b) \log p(a,b) = \sum_{a = b} p(a, b) \log p(a,b)$$
where in the second equality we applied the assumption ...
- 1,764
4
votes
Joint entropy of a random variable with itself
But what is $H(AB)$ if A and B are not independent, for example, if they are identical?
dmh has posted the calculation which demonstrates that two identical distributions have the same entropy as ...
- 8,210
Only top scored, non community-wiki answers of a minimum length are eligible