# Is there a symmetric alternative to Kullback-Leibler divergence?

I have two samples of probability distributions that I would like to compare. I have previously heard about the Kullback-Leibler divergence, but reading up on this it seems like its non-symmetricity makes it more suitable for comparing a sample to a model, rather than comparing two samples. What would you propose that I use instead, or maybe the KL divergence actually is a good choice?

• You can use the symmetric KL divergence, $D_{KL}(P, Q) + D_{KL}(Q, P)$. See here for more options: en.wikipedia.org/wiki/… . There are also many options that aren't based on the KL divergence, like statistical distance $\frac{1}{2}\int |P-Q|$, which is a metric. Even more options are here: en.wikipedia.org/wiki/Statistical_distance . – Reinstate Monica Nov 18 '14 at 21:42
• – Did Nov 18 '14 at 22:20

Beyond the symmetric KL-divergence, Information Theoretic Learning presented several symmetric distribution "distances". The idea is just to realize that pdfs are like any other functions in a L2-space. Thus, you can calculate the Euclidian distance $\int_x(p(x)-q(x))^2dx$, Cauchy-Schwarz distance, etc. There are even approximations to these distances directly from data, using Parzen Windows. Check the link above for more.