The KL-divergence is defined as:
$D_{KL} (p(x) \parallel q(x)) = \sum_x p(x) \log \frac{p(x)}{q(x)} $
If $A$ and $B$ are discrete variables, does it make sense to calculate $D_{KL}(p(A, B) \parallel p(A))$? Namely, the divergence between the joint distribution $p(A, B)$, and the marginal distribution $p(A)$. Or must they be of the same type (i.e. defined over the same and all variables)?
I tried, and I found this:
$ \begin{align} D_{KL}(p(A, B) \parallel p(A)) &= \sum_{a \in A} \sum_{b \in B} p(a, b) \log \frac {p(a, b)}{p(a)} \\ &= ~... \\ &= H\,(B \mid A) \end{align} $
Here, $H$ is the Shannon entropy.
Now, if needed, we could make the second distribution to be the same type, by defining $D_{KL}(p(A, B) \parallel q(A, B))$, where we simply say that $q(A, B) = Pr(A)$. Would it then be okay to calculate $D_{KL}(p(A, B) \parallel q(A))$?
In the book "Elements of Information Theory", by Cover and Thomas, it says that $D_{KL}(p(x) \parallel q(x)) = \infty $ if the distribution $q$ doesn't define a probability value for every symbol that $p$ defines.