Prokhorov metric vs. total variation norm Let $(S,d)$ be a metric space and let $\mathcal P(S)$ denote the space of Borel probability measures on $S$ endowed with the Prokhorov metric $\pi:\mathcal P(S)\times \mathcal P(S)\to \mathbb R_+$ given by
$$
  \pi(P,Q):=\inf\{\varepsilon\geq 0:P(F)\leq Q(F^\varepsilon)+\varepsilon \text{ for all closed } F\subset S\}
$$
where the $\varepsilon$-inflation of a set is given by  $  F^{\varepsilon} = \{x\in S:d(x,F)<\varepsilon\}.$
Another useful metric on $\mathcal P(S)$ is induced by the total variation norm, i.e. 
$$
  \rho(P,Q):=\sup\limits_{A\in \mathfrak B(S)}|P(A) - Q(A)|
$$
where $\mathfrak B(S)$ is the Borel $\sigma$-algebra on $(S,d)$. I wonder if there are any interesting relations between these two metrics, $\pi$ and $\rho$. In particular, I know that convergence in $\rho$ implies the weak convergence and hence if $S$ is separable than it implies the convergence in $\pi$. 
I wonder, however, if under some additional assumptions it is possible to derive some non-trivial bounds on $\rho$ if I know upper bounds on $\pi$. Or at least, if it is possible to upper-bound $|P(F) - Q(F)|$ for closed $F$ using $\pi$.
 A: Generally, knowing things about $\pi$ cannot tell you much about $\rho$; weak convergence is much easier to achieve than total variation convergence.  
For example, if $S$ is any non-discrete space, $x$ is a limit point, and $x_n \to x$ is a sequence with $x_n \ne x$ for all $n$, then the point masses $\delta_{x_n}$ satisfy $\pi(\delta_{x_n}, \delta_x) \to 0$.  (Indeed, $\pi(\delta_{x_n}, \delta_x) = d(x_n, x)$.)  But we have $\rho(\delta_{x_n}, \delta_x) = 1$ for every $n$.
A: Here is a simple inequality relating the Prohorov metric $\pi$ and the total variation norm $\|\;\|_{TV}$for probability measures:
Suppose $\mu$ and $\nu$ are probability measures on a metric space $(S,d)$. For any Borel set $A\subset S$,
\begin{aligned}
\mu(A)&=\mu(A)-\nu(A)+\nu(A)\leq \|\mu-\nu\|_{TV}+\nu(A) \\
&\leq \|\mu-\nu\|_{TV} + \nu\big(A^{\|\mu-\nu\|_{TV}}\big)
\end{aligned}
whence we obtained that
$$\pi(\mu,\nu)\leq\|\mu-\nu\|_{TV}
$$

Another relevant metric is the Kantorovih-Rubinstein metric
$$\beta(\mu,\nu)=\sup\left\{\Big|\int f(d\mu-d\nu)\Big|: f\in\operatorname{Lip}_1\right\}
$$
where $\operatorname{Lip}_1$ is the space of Lipschitz functions on $S$ such that $\|f\|_{BL}=\|f\|_\infty+\|f\|_L\leq1$
where $\|f\|_L=\sup_{x\neq y}\frac{|f(x)-f(y)|}{d(x,y)}$. It is a well know fact that $\beta$ and $\pi$ produce the same topology on the space of probability measures, namely the weak topology $\sigma(\mathcal{M}(S),\mathcal{C}_b(S))$ restricted to $\mathcal{M}^+_1(S)$ the space of probabilities on $(S,d)$. The following is also well known:
\begin{aligned}
\frac12\beta(\mu,\nu)&\leq \pi(\mu,\nu)\leq\Big(\frac32\beta(\mu,\nu)\Big)^{1/2}\\
\beta(\mu,\nu)&\leq \|\mu-\nu\|_{TV}
\end{aligned}
Dudley's book Real Analysis and Probability chapter 11  is a good reference.
A: On metric spaces, the Prokhorov metric can always be bounded by the total variation metric. If $S$ is finite, you can bound the total variation metric by the Prokhorov metric via the Wasserstein metric. 
More metrics and their relations (including the ones above) are nicely summariced in Gibbs A.L. and Su F.E., On Choosing and Bounding Probability Metrics,  International Statistical Review 70 (2002), pp. 419-435.
