# Maximum entropy distribution given second order marginals

Let $p(x,y,z)$ be a probability distribution over 3 variables (suppose them discrete, but it shouldn't matter). I know that the distribution with maximal entropy which preserves the first order marginals $p(x),p(y),p(z)$ is simply the product $p(x)p(y)p(z)$.

What is instead the distribution with maximal entropy which preserves these marginals: $p(x,y),p(x,z),p(y,z)$? Or if there is no closed form, can I define it implicitly?

Thanks.

• My intuition tells me you will only be able to preserve one of $p(x,y)$, $p(x,z)$ or $p(y,z)$ at a time unless you take the full joint probability. – Dan Mar 11 '15 at 13:48
• Why? Just take a lower entropy. – geodude Mar 11 '15 at 13:52
• I'm not sure I follow, could you give an example? – Dan Mar 11 '15 at 13:56
• There always exist a distribution which has all the same double marginals, namely the original $p$. And possibly some other...among which we take the one with the highest entropy. If it were impossible to find others, it would mean that the double marginals determine $p$ uniquely, which is absurd. – geodude Mar 11 '15 at 14:02
• Why is it absurd that all three double marginals taken together uniquely define $p$? I'm trying to find an example where that is not the case but so far I've been unsuccessful. – Dan Mar 11 '15 at 14:05