How does one find the maximum entropy joint distribution of two random variables X and Y given their marginal probability mass functions?
- I have the marginals, meaning p(x) and p(y) are fixed.
- The entropy is maximized when the distribution is even (p(x,y) = 1/n for all x,y), but it can't be even due to the marginals.
- The joint distribution is only the product of the marginals when X and Y are independent.
- The KL Divergence looks handy, but I can't use it to prove independence (zero mutual info) if I only know the marginals.
Does anyone know what I'm missing?