# Similarity between a set of points and a given probability distribution

Given a set of points $D=\{x_1,x_2,\cdots,x_n\}$, I want to know if the points are distributed according to some distribution $\mathbb P$. Ideally, I would like to have a similarity or distance metric $\mathcal d(\mathbb P, D)$, and implement it in a code. What is a convenient metric for this task?

• A possibility would be to use the Kullback–Leibler divergence (which however is not a metric). – derpy Oct 18 '17 at 20:34
• You could use the likelihood of the points, if you know their joint distribution under $\mathbb{P}.$ This isn't a distance, but there is some sense to the idea that if $D$ has a higher likelihood under $\mathbb{P}$ than under $\mathbb{P}',$ we should expect it to have been generated from $\mathbb{P}$ rather than from $\mathbb{P}'.$ This is the rough idea of the maximum likelihood estimator. – RideTheWavelet Oct 18 '17 at 20:34
• @derpy But how could I apply it between a finite set of points and, say, the normal distribution? – Michael Oct 18 '17 at 20:40
• @RideTheWavelet Yeah, that's the objective function I was using so far, but I realized what I really need is the distribution of the data and how close it is to another distribution, rather than it's likelihood. – Michael Oct 18 '17 at 20:42
• @Michael You should define your empirical distribution to be $p(x) = \frac{1}{n} \sum_i \delta(x-x_i)$ and compute $D_{KL}(g||p)$ (with $g$ the normal distribution). However, this is basically a a maximum likelihood principle, so I don't know if that's what you're looking for. – derpy Oct 18 '17 at 20:51