Given a set of points $D=\{x_1,x_2,\cdots,x_n\}$, I want to know if the points are distributed according to some distribution $\mathbb P$. Ideally, I would like to have a similarity or distance metric $\mathcal d(\mathbb P, D)$, and implement it in a code. What is a convenient metric for this task?

  • $\begingroup$ A possibility would be to use the Kullback–Leibler divergence (which however is not a metric). $\endgroup$ – derpy Oct 18 '17 at 20:34
  • $\begingroup$ You could use the likelihood of the points, if you know their joint distribution under $\mathbb{P}.$ This isn't a distance, but there is some sense to the idea that if $D$ has a higher likelihood under $\mathbb{P}$ than under $\mathbb{P}',$ we should expect it to have been generated from $\mathbb{P}$ rather than from $\mathbb{P}'.$ This is the rough idea of the maximum likelihood estimator. $\endgroup$ – RideTheWavelet Oct 18 '17 at 20:34
  • $\begingroup$ @derpy But how could I apply it between a finite set of points and, say, the normal distribution? $\endgroup$ – Michael Oct 18 '17 at 20:40
  • $\begingroup$ @RideTheWavelet Yeah, that's the objective function I was using so far, but I realized what I really need is the distribution of the data and how close it is to another distribution, rather than it's likelihood. $\endgroup$ – Michael Oct 18 '17 at 20:42
  • 1
    $\begingroup$ @Michael You should define your empirical distribution to be $ p(x) = \frac{1}{n} \sum_i \delta(x-x_i) $ and compute $ D_{KL}(g||p) $ (with $ g $ the normal distribution). However, this is basically a a maximum likelihood principle, so I don't know if that's what you're looking for. $\endgroup$ – derpy Oct 18 '17 at 20:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.