Name or intuition for this quantity similar to mutual information?

The mutual information between random variables $$X$$ and $$Y$$ is defined (in terms of their joint PDF or PMF $$p$$) [Wikipedia]

$$I(X; Y) = \mathbb{E}_{p(X, Y)}\left[ \log \frac{p(X, Y)}{p(X)p(Y)} \right].$$

Equivalently we can write $$I(X; Y) = \mathbb{E}_{p(X, Y)}\left[ \log {p(X, Y)} \right] - \mathbb{E}_{p(X, Y)}\left[\log {p(X)p(Y)} \right].$$

When doing some calculations, I got the following quantity, which is the same but swapping the independent and joint distributions in the right-hand side. Is there a name for it, or an intuitive interpretation of it?

$$F(X, Y) = \mathbb{E}_{p(X, Y)} \left[ \log p(X, Y) \right] - \mathbb{E}_{p(X)p(Y)} \left[ \log p(X, Y) \right]$$

• What does $\mathbb{E}_{p(X)p(Y)}$ mean? I know that $\mathbb{E}_{p(X,Y)}$ means the expected value.
– Joe
Commented Oct 22, 2019 at 11:33
• The expected value, but over the marginal distributions of X and Y instead of over their joint. $$\mathbb{E}_{p(X)p(Y)}[f(X, Y)] = \mathbb{E}_{p(X)} \mathbb{E}_{p(Y)} f(X, Y)$$ Commented Oct 22, 2019 at 14:47
• If you compute $\mathbb{E}_{p(Y)}f(X,Y)$ as a function of $X$, then compute the expected value of that function over the distribution of $X$, doesn’t it give you $\mathbb{E}_{p(X,Y)}f(X,Y)$?
– Joe
Commented Oct 22, 2019 at 15:37
• No, it does not. That would only be true if you first computed $\mathbb{E}_{p(Y | X)}f(X, Y)$ as a function of $X$. Commented Oct 23, 2019 at 10:05
• You should ask this on Cross Validated. You’ll probably get more responses.
– Joe
Commented Oct 23, 2019 at 19:21

Mutual information can be written as the Kullback-Leibler (KL) divergence between the joint distribution $$p(X,Y)$$ and the product distribution $$p(X)p(Y)$$, $$I(X;Y) = D_{KL}(p(X,Y)\Vert p(X)p(Y))$$
There is also a symmetrized version of KL divergence, sometimes called Jeffreys divergence or J-divergence, $$D_J(p\Vert q) = D_{KL}(p\Vert q) + D_{KL}(q\Vert p)$$ (see e.g. https://arxiv.org/pdf/1009.4004.pdf).
Your expression $$F(X,Y)$$ is the Jeffreys divergence between the joint distribution $$p(X,Y)$$ and product distribution $$p(X)p(Y)$$: \begin{align} D_J(p(X,Y)\Vert p(X)p(Y)) &= I(X;Y) + D_{KL}(p(X)p(Y)\Vert p(X,Y))\\ & = H(X) + H(Y) - H(X,Y)- H(X)-H(Y) - \mathbb{E}_{p(X)p(Y)}[\log p(X,Y)] \\ & = \mathbb{E}_{p(X,Y)}[\log p(X,Y)] - \mathbb{E}_{p(X)p(Y)}[\log p(X,Y)] \end{align}
The "reverse" KL term $$D_{KL}(p(X)p(Y)\Vert p(X,Y))$$ is sometimes called lautum information, and is described in the following: