# Minimal I-divergence and marginals

Let $$Q_1, \dots, Q_n$$ be arbitrary distributions over the finite sets $$X_1, \dots, X_n$$, and $$P$$ be an arbitrary distribution over $$X_1 \times \cdots \times X_n$$ with marginals $$P_1, \dots, P_n$$.

How can I prove that $$D(P||Q_1\times \cdots \times Q_n)=D(P||P_1\times \cdots \times P_n) + \sum_{i=1}^n D(P_i||Q_i),$$ and conclude that among the distributions $$P$$ with marginals $$P_1, \dots, P_n$$ the I-divergence $$D(P||Q_1\times \cdots \times Q_n)$$ is minimal if $$P = P_1\times \cdots \times P_n$$?

\begin{align} D(P\,||\,Q_1 \times ... \times Q_n) &= \sum\limits_{x_1, ..., x_n} p(x_1, ..., x_n) \log\left(\frac{p(x_1, ..., x_n)}{q(x_1) \times ... \times q(x_n)}\right) \\ &= \sum\limits_{x_1, ..., x_n} p(x_1, ..., x_n) \log\left(\frac{p(x_1)\times ... \times p(x_n)}{q(x_1) \times ... \times q(x_n)}\right) \\ &\quad + \sum\limits_{x_1, ..., x_n} p(x_1, ..., x_n) \log\left(\frac{p(x_1, ..., x_n)}{p(x_1) \times ... \times p(x_n)}\right) \\ &= \sum\limits_{i=1}^n \sum\limits_{x_i}p_i(x_i) \log\left(\frac{p(x_i)}{q(x_i)}\right) + D(P\,||\,P_1 \times ... \times P_n) \\ &= \sum\limits_{i=1}^n D(P_i\,||\,Q_i) + D(P\,||\,P_1 \times ... \times P_n). \end{align}