# KL divergence between 2 joint probability distribution

Consider the following setting : $$Q(x,z)= q_{data}(x)q_{\phi}(z|x) \;\;\;\text{and} \;\;\; P(x,z)= p(z)p_{\theta}(x|z) = p_{\theta}(x)p_{\theta}(z|x)$$ I want to compute $$KL(Q||P)$$. Here is how I am computing this KL $$KL(Q||P) = \int_x\int_z q_{data}(x)q_{\phi}(z|x) \log\big(\frac{q_{data}(x)q_{\phi}(z|x)}{p_{\theta}(x)p_{\theta}(z|x)}\big)dxdz$$ $$= \int_x\int_z q_{data}(x)q_{\phi}(z|x) \log\big(\frac{q_{data}(x)}{p_{\theta}(x)}\big)dxdz + \int_x\int_z q_{data}(x)q_{\phi}(z|x) \log\big(\frac{q_{\phi}(z|x)}{p_{\theta}(z|x)}\big)dxdz \;\;\;\; (1)$$ The first term of the right-hand side (RHS) of the eq.1 is not posing problem : $$\int_x\int_z q_{data}(x)q_{\phi}(z|x) \log\big(\frac{q_{data}(x)}{p_{\theta}(x)}\big)dxdz=KL(q_{data}(x)||p_{\theta}(x))$$ ( I can transform the double integral into a single one (over x) cause the only term depending on z (i.e. $$q_{\phi}(z|x)$$) has a nice integration over z (i.e. $$\int_z q_{\phi}(z|x) dz = 1$$) )

Now the second RHS term of eq.1 :

$$\int_x\int_z q_{data}(x)q_{\phi}(z|x) \log\big(\frac{q_{\phi}(z|x)}{p_{\theta}(z|x)}\big)dxdz = \int_x q_{data}(x) \Big(\int_z q_{\phi}(z|x)\log\big(\frac{q_{\phi}(z|x)}{p_{\theta}(z|x)}\big)dz\Big)dx$$ $$= \mathbb{E}_{q_{data(x)}}[KL(q_{\phi}(z|x)||p_{\theta}(z|x))]$$

But in this article [1] (which is great btw), they are saying that this second term is equal to $$KL(q_{\phi}(z|x)||p_{\theta}(z|x))$$... So slightly different from what I have. I suspect they are applying the same trick than in the first term of eq.1, which is on my mind not applicable cause we cannot isolate the integral $$\int_xq_{data}(x)dx$$ from the other terms as they depend both on x and y.

So here is are my 2 questions :

• Am I right thinking that this trick (i.e. the isolation of the nice integral $$\int_xq_{data}(x)dx$$) is not applicable in the second term of the RHS of eq.1 ?
• Am I right thinking that this second term of the RHS of eq.1 is equal to $$\mathbb{E}_{q_{data(x)}}[KL(q_{\phi}(z|x)||p_{\theta}(z|x))]$$ and not to $$KL(q_{\phi}(z|x)||p_{\theta}(z|x))$$ ?

I know it seems to be a minor detail... but it is bugging my mind since the last 2 days.. and obviously it has consequence on the rest of the calculation.