7
$\begingroup$

I have two samples of probability distributions that I would like to compare. I have previously heard about the Kullback-Leibler divergence, but reading up on this it seems like its non-symmetricity makes it more suitable for comparing a sample to a model, rather than comparing two samples. What would you propose that I use instead, or maybe the KL divergence actually is a good choice?

$\endgroup$
3
$\begingroup$

Beyond the symmetric KL-divergence, Information Theoretic Learning presented several symmetric distribution "distances". The idea is just to realize that pdfs are like any other functions in a L2-space. Thus, you can calculate the Euclidian distance $\int_x(p(x)-q(x))^2dx$, Cauchy-Schwarz distance, etc. There are even approximations to these distances directly from data, using Parzen Windows. Check the link above for more.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.