Is there any official (i.e., to be found in probability books) metric for the distance between two probability measures, defined only on a subset of their support?

Take, for example, the total variation distance: $$TV(\mu,\nu)=\sup_{A\in\mathcal{F}}|\mu(A)-\nu(A)|.$$

If $X$ and $Y$ are two real positive continuous random variables with densities $f_X$ and $f_Y$, then their total variation distance is, if I understand correctly: $$TV(\mu_X,\mu_Y)=\int_{0}^{\infty}|f_X(z)−f_Y(z)|dz.$$

Would it make any sense to calculate a quantity, for $\tau>0$, let's call it partial distance, like this: $$PV(\mu_X,\mu_Y;\tau)=\int_{\tau}^{\infty}|f_X(z)−f_Y(z)|dz\;\;\;?$$

If this does not make any sense (sorry, I really cannot tell, as I am not that good with measure theory...), can anyone think of a measure that would make sense?

What I want to use this for is to compare the closeness of two PDFs (or other functions describing a distribution: CDF, CCDF...) $f_X(t)$, $f_Y(t)$ to a third one $f_Z(t)$. I know that both $f_X$ and $f_Y$ "eventually" ($t\to\infty$) converge to $f_Z$, but I would like to show that one of them gets closer, sooner than the other one...


I gave you this same answer over at MathOverflow. Looking at your response to Michael Chernick, you probably do want to consult Dudley, as the Prohorov metric and its follow-up in Proposition 11.3.2 refer directly to metrics on random variables, which could be defined for the tail only as you request.

You may want to check out Real Analysis and Probability by R. M. Dudley (2002, Cambridge University Press). Chapters 9-11 discuss several metrics on probability measures and random variables (laws), and since restricting your support would be equivalent to some random variable on the measure, you should be able to use something like the metrics discussed in section 11.3 in particular.

  • $\begingroup$ Same comment on MathOverflow: I will check out the book as soon as I can get my hands on it. However, looking at the Google preview of Chapter 11, I am not sure I will be able to draw very much from it. As I've said in the question, I am not very well equipped to understand all the technical finesse of measure theory. This is why I was hoping for an answer which relates more directly to the difference between two densities (or CDFs or CCDFs) at the tail... $\endgroup$ – miladydesummer Jul 26 '12 at 15:42
  • $\begingroup$ Got my hands on the book and read the chapter in question. Can you tell me why you think that particular measure is more suitable for my problem than others mentioned (total variation, Kullback-Leibler)? The Prokhorov metric also spans all the subsets of the support... If I understand correctly, you suggest to create some new random variables on the same probability space, which would be defined for the tail only (which I'm assuming should be possible regardless of the distance metric). Is that even possible with a probability measure on the positive reals, like I have? $\endgroup$ – miladydesummer Jul 26 '12 at 21:30
  • $\begingroup$ It's more suitable only in the sense that it's proven to BE a metric, which you asked for. I'm an actuary by training, so I'm familiar with ad hoc ways of measuring tail "thickness", and the difference between tails would fit into that category. Your TV integral limited to your tail seems fine: it's just a limited form of of the $\beta$ metric from Dudley in that it's only measured with respect to one Lipschitz function (i.e. 1) rather than a nice "unit ball" of them. As for a new tail RV, just hit your original RV with the appropriate indicator function to zero out what you don't want. $\endgroup$ – user452 Jul 28 '12 at 0:39
  • $\begingroup$ Great! Thanks for the explanation, this is one thing that I was very curious about (the difference between integrating on the whole domain vs only part of it). $\endgroup$ – miladydesummer Jul 28 '12 at 6:54

In statistics there are several different measures of distance that can be used depending on application. When determining whether or not two multivariate normal distributions with the same covariance matrix have the same mean vector the Mahalanobis distance is used. This can be applied in discriminant analysis or cluster analysis. Link:http://en.wikipedia.org/wiki/Mahalanobis_distance

From the perspective of information theory people often use the Kullback-Leiber distance. Link:http://en.wikipedia.org/wiki/Kullback%E2%80%93Leibler_divergence The following link gives these measures and several more:


  • 1
    $\begingroup$ I am aware of both those metrics. The Mahalanobis distance seems to be rather directed to sample sets, whereas I am looking for an "analytical" metric for continuous distributions. The Kullback-Leibler metric has exactly the same problem as the total variation example I gave in the question (i.e., it is an integral from $-\infty$ to $+\infty$ for real r.v.'s). The question is whether it still makes mathematical sense to take this integral from $\tau$ to $+\infty$ instead. And if someone thinks it does make sense, how do they justify it? $\endgroup$ – miladydesummer Jul 26 '12 at 14:56
  • $\begingroup$ Sorry I missed the part about looking only at the tail of the distributions. Certainly difference in the tails is important in extreme value theory and the t distribution with low degrees of freedom is shaped like the normal but with heavier tails that approach the normal as the degrees of freedom increase. $\endgroup$ – Michael Chernick Jul 26 '12 at 15:20
  • $\begingroup$ I could see such a measure being useful for comparing symmetric distributions, normal vs t vs Cauchy and a one sided measure for distributions like the chi square, lognormal and F. $\endgroup$ – Michael Chernick Jul 26 '12 at 15:26

Something like this is what I was looking for:

"A measure of discrimination between two residual life-time distributions and its applications" by Nader Ebrahimi and S.N.U.A. Kirmani from the Annals of the Institute of Statistical Mathematics, Volume 48, Number 2 (1996), 257-265

Can be downloaded from:

(official) http://www.springerlink.com/content/r186240907v85964/

(maybe official?) http://www.ism.ac.jp/editsec/aism/pdf/048_2_0257.pdf

EDIT: I realize the proposal in the paper is not a metric (because not symmetric), but just to give an example of what I have in mind... Would a similar adaptation be possible for other measures of distance which are metrics?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.