# Prokhorov metric vs. total variation norm

Let $(S,d)$ be a metric space and let $\mathcal P(S)$ denote the space of Borel probability measures on $S$ endowed with the Prokhorov metric $\pi:\mathcal P(S)\times \mathcal P(S)\to \mathbb R_+$ given by $$\pi(P,Q):=\inf\{\varepsilon\geq 0:P(F)\leq Q(F^\varepsilon)+\varepsilon \text{ for all closed } F\subset S\}$$ where the $\varepsilon$-inflation of a set is given by $F^{\varepsilon} = \{x\in S:d(x,F)<\varepsilon\}.$

Another useful metric on $\mathcal P(S)$ is induced by the total variation norm, i.e. $$\rho(P,Q):=\sup\limits_{A\in \mathfrak B(S)}|P(A) - Q(A)|$$ where $\mathfrak B(S)$ is the Borel $\sigma$-algebra on $(S,d)$. I wonder if there are any interesting relations between these two metrics, $\pi$ and $\rho$. In particular, I know that convergence in $\rho$ implies the weak convergence and hence if $S$ is separable than it implies the convergence in $\pi$.

I wonder, however, if under some additional assumptions it is possible to derive some non-trivial bounds on $\rho$ if I know upper bounds on $\pi$. Or at least, if it is possible to upper-bound $|P(F) - Q(F)|$ for closed $F$ using $\pi$.

On metric spaces, the Prokhorov metric can always be bounded by the total variation metric. If $S$ is finite, you can bound the total variation metric by the Prokhorov metric via the Wasserstein metric.

More metrics and their relations (including the ones above) are nicely summariced in Gibbs A.L. and Su F.E., On Choosing and Bounding Probability Metrics, International Statistical Review 70 (2002), pp. 419-435.

• Thanks a lot for such a nice reference! Just one comment, did you mean that if $S$ is finite than Prokhorov metric can be bounded by the TV one via the Wasserstein metric? – S.D. Nov 28 '12 at 13:00
• Actually, the first statement was incorrect ($\pi\leq \rho$, but $\rho \leq c \pi$ only for finite $S$); fixed now. Thanks for pointing this out! – Christian Clason Nov 28 '12 at 13:04
• I also misread the direction of the arrows, thanks a lot! – S.D. Nov 28 '12 at 13:13

Generally, knowing things about $\pi$ cannot tell you much about $\rho$; weak convergence is much easier to achieve than total variation convergence.

For example, if $S$ is any non-discrete space, $x$ is a limit point, and $x_n \to x$ is a sequence with $x_n \ne x$ for all $n$, then the point masses $\delta_{x_n}$ satisfy $\pi(\delta_{x_n}, \delta_x) \to 0$. (Indeed, $\pi(\delta_{x_n}, \delta_x) = d(x_n, x)$.) But we have $\rho(\delta_{x_n}, \delta_x) = 1$ for every $n$.

• Thanks, Nate - I guess it answers also my point about closed sets. Also, here I think $\rho(\delta_{x_n},\delta_x) = 1$ as I defined it as a half-total variation (often done for probability measures) – S.D. Nov 28 '12 at 13:45
• @S.D.: You're right, it should be 1. I fixed it. – Nate Eldredge Nov 28 '12 at 14:59

Here is a simple inequality relating the Prohorov metric $$\pi$$ and the total variation norm $$\|\;\|_{TV}$$for probability measures: Suppose $$\mu$$ and $$\nu$$ are probability measures on a metric space $$(S,d)$$. For any Borel set $$A\subset S$$,

\begin{aligned} \mu(A)&=\mu(A)-\nu(A)+\nu(A)\leq \|\mu-\nu\|_{TV}+\nu(A) \\ &\leq |\mu-\mu\|_{TV} + \nu\big(A^{\|\mu-\nu\|_{TV}}\big) \end{aligned} whence we obtained that $$\pi(\mu,\mu)\leq\|\mu-\nu\|_{TV}$$

Another relevant metric is the Kantorovih-Rubinstein metric $$\beta(\mu,\nu)=\sup\left\{\Big|\int f(d\mu-d\nu)\Big|: f\in\operatorname{Lip}_1\right\}$$ where $$\operatorname{Lip}_1$$ is the space of Lipschitz functions on $$S$$ such that $$\|f\|_{BL}=\|f\|_\infty+\|f\|_L\leq1$$ where $$\|f\|_L=\sup_{x\neq y}\frac{|f(x)-f(y)|}{d(x,y)}$$. It is a well know fact that $$\beta$$ and $$\pi$$ produce the same topology on the space of probability measures, namely the weak topology $$\sigma(\mathcal{M}(S),\mathcal{C}_b(S))$$ restricted to $$\mathcal{M}^+_1(S)$$ the space of probabilities on $$(S,d)$$. The following is also well known: \begin{aligned} \frac12\beta(\mu,\nu)&\leq \pi(\mu,\nu)\leq\Big(\frac32\beta(\mu,\nu)\Big)^{1/2}\\ \beta(\mu,\nu)&\leq \|\mu-\nu\|_{TV} \end{aligned}

Dudley's book Real Analysis and Probability chapter 11 is a good reference.