# Does this quantity have a name in information theory?

The Kullback-Leibler divergence between two (discrete) probability distributions is defined as $$D_{KL}(P\|Q) = \sum_i p_i \log \frac{p_i}{q_i},$$ where $p_i$ is the probability that $P$ assigns to the event $i$, and $q_i$ is the probability assigned by $Q$.

I know that the quantity $D_{KL}(P\|Q) + D_{KL}(Q\|P)$ (symmetrised Kullback-Leibler divergence) is sometimes used, because it is symmetric and thus behaves more like a distance between the two distributions. But does anyone know of a case where the quantity $$\sum_i (p_i-q_i) \log \frac{p_i}{q_i}$$ is used, and whether it has a standard name? I ask because it came up in some statistical mechanics work I'm doing and I want to know if it has an interpretation in terms of information theory, or any particularly interesting known properties.

-
Don't we have $(p - q)\log \frac pq = p \log\frac pq + q \log \left(\frac pq\right)^{-1} = p\log\frac pq + q \log \frac qp$, so the quantity you are looking at is the symmetrised KL-divergence? –  martini Mar 19 '12 at 11:20
D'oh - yes, you're right. I kept getting confused with the signs and at first thought it was $D_{KL}(P\|Q)-D_{KL}(Q\|P)$ (hence mentioning the symmetrised KL-divergence in the question). Then I realised that wasn't right, but I didn't spot that it's just equal to the symmetrised KL-divergence. So this is actually a pretty silly question - not sure whether I should just delete it. –  Nathaniel Mar 19 '12 at 11:29
I'll post an answer, so if you choose to keep, you can accept. –  martini Mar 19 '12 at 12:01

## 1 Answer

Since \begin{align*} \sum_i (p_i - q_i) \log \frac{p_i}{q_i} &= \sum_i p_i \log \frac{p_i}{q_i} - \sum_i q_i \log \frac{p_i}{q_i}\\ &= \sum_i p_i \log \frac{p_i}{q_i} + \sum_i q_i \log \left(\frac{p_i}{q_i}\right)^{-1}\\ &= D_{KL}(P||Q) + D_{KL}(Q||P) \end{align*} the quantity in question is the symmetrised KL-divercence,

AB,

-