Closure in the Space of Probability Measures with the Prohorov metric

I have seen this result stated countless times: assume the metric space $(\theta,d)$ is separable; then $(\theta,d)$ is complete if and only if the space $(\mathcal{P}(\Theta),\rho)$ (the space of probability measures, taken with the Prohorov metric--which is equivalent to the weak* topology) is complete. See, for instance,

• Billingsley, Convergence of Probability Measures (1968), pg 240
• W. Whitt, Weak Convergence of Probability Measures on the Function Space $C[0,\infty)$, Annals of Math. Stat 41 (1970), Corollary 2
• http://www.math.leidenuniv.nl/~vangaans/jancol1.pdf , Theorem 9.2

But if this result holds (say, for $\Theta = \mathbb{R}$), then $(\mathcal{P}(\Theta),\rho)$ is weak* closed, which is false (see, e.g., milanmerkle.com/documents/radovi/WEACO2a.pdf , Section 5.4; a similar discussion has been had on these boards: Is the set of all probability measures weak*-closed?)

The proof of the first result typically relies on Prohorov's Theorem: take a Cauchy sequence $\{P_{n}\}$, show that the sequence is tight, therefore it is relatively sequentially compact. But in all the cases mentioned above, the authors use relative sequential compactness to conclude that the sequence must have a convergent subsequence in the space of probability measures (rather than the closure of that space). This seems to be the error, but the result is stated so ubiquitously that I feel I may be missing something....

UPDATE: I have located another related question: Tightness of a sequence of probability measures and weak convergence of a subsequence In Billingsley's proof (of my result), we take a Cauchy sequence $\{P_{n}\}$ and show that the sequence is tight, which is taken to prove the convergence of a subsequence. But this result seems incorrect, since tightness is only sufficient to prove the relative compactness of the set of measures in the sequence (by Prohorov's Theorem).

• Why "if the result holds, then $(\cal P(\Theta),\rho)$ is weak$^*$ closed"? – Davide Giraudo Nov 29 '12 at 15:00
• If $(\mathcal{P}(\Theta),\rho)$ is weak* complete, then it must be weak* closed – Clay Nov 29 '12 at 15:41
• You mean the weak star topology of which set? – Davide Giraudo Nov 29 '12 at 20:37
• I think this may be very closely related to the problem I am having... It is trivial that $(\mathcal{P}(\Theta),\rho)$ is weak* closed in $(\mathcal{P}(\Theta),\rho)$ (in the subspace topology). But perhaps $(\mathcal{P}(\Theta),\rho)$ is not weak* closed in the larger space of measures? It seems possible that this may explain the divergent results obtained above, but it is not clear (to me). – Clay Nov 29 '12 at 21:02

The issue is basically that on the space of probability measures, the topology induced by the Prohorov metric is not always the weak* topology. Basically, referring to convergence in the Prohorov metric as "weak convergence" of probability measures is somewhat unfortunate in this sense. If this claim is true, your example of $\Theta = \mathbb{R}$ does not yield a contradiction after all.

To put this into perspective, note the following. Let $C_b(\Theta)$ denote the set of bounded continuous functions on $\Theta$. Define $T_f:\mathcal{P}(\Theta)\to\mathbb{R}$ by $T_f(\mu) = \int f \; d\mu$. Two definitions:

1. A sequence of of probability measures $(\mu_n)$ converges to $\mu$ in the Prohorov metric if and only if $T_f(\mu_n)$ converges to $T_f(\mu)$ for all $f\in C_b(\Theta)$.
2. When $\mathcal{P}(\Theta)$ is a subset of the dual of $C_b(\Theta)$, a sequence of probability measures $(\mu_n)$ converges to $\mu$ in the weak* topology if and only if $T_f(\mu_n)$ converges to $T_f(\mu)$ for all $f\in C_b(\Theta)$.

So, when $\mathcal{P}(\Theta)$ is a subset of the dual of $C_b(\Theta)$, convergence in the Prohorov metric and weak* convergence are the same. However, this is not always the case.

If $(\Theta,d)$ is compact, it holds that $C(\Theta) = C_c(\Theta) = C_b(\Theta)$, i.e. the spaces of continuous functions, continuous functions with compact support and bounded continuous functions are the same, and furthermore, $C(\Theta)' = rca(\Theta)$, where $rca(\Theta)$ is the set of regular Borel measures on $(\Theta,d)$, and so $\mathcal{P}(\Theta)\subseteq rca(\Theta) = C_b(\Theta)$. Thus, in this case, convergence in the Prohorov metric and weak* convergence is the same. However, when $(\Theta,d)$ is not compact, the duality relationships do not always hold. I don't really have a counterexample at hand, but I'm confident that you can find one somewhere.

The above is basically one of the good reasons for having a theory of "weak convergence" for probability measures which is not just a reference to weak* convergence. For probability theory, our main interest is measures which are not supported on compact sets, and therefore we need a convergence theory which holds for such measures. The Prohorov metric and the associated convergence concept based on $C_b(\Theta)$ turns out to be a good solution.