# Weak convergence of probability measure

I am working on a problem. Show that for each probability measure $\mu$, there exists probability measure $\mu_n$ with finite support such that $\mu_n$ converges weakly to $\mu$. I am thinking about the empirical measure, which has the distribution function $F_n(t)=1/n\sum 1(X_i\le t)$. So from LLN, we have $F_n(t)\rightarrow F(t)$ for each fixed $t$ a.s. But the convergence here is almost surely. So does this still means $F_n$ converges to $F$ weakly?

-
Is $\mu$ defined on what kind of set? Is it defined over the reals, over some compact or locally compact topological space? – André Caldas Apr 17 '12 at 4:23

You got the idea. Let $F_n(t,\omega):=\frac 1n\sum_{j=1}^n\chi_{(-\infty,t]}(X_k(\omega))$, where $X_j$ are independent random variable of law $\mu$. By Glivenko-Cantelli theorem, also known as fundamental theorem of statistics, we know that for almost all $\omega\in\Omega$, we have $$\sup_{t\in\mathbb R}|F_n(t,\omega)-F(t)|\to 0.$$

Fix one of these $\omega$, and let $\mu_n$ the probability measure associated with the cumulative distribution function $F_n(t)=F_n(t,\omega)$. Since $F_n(t)\to F(t)$ at all points of continuity of $F$, we have that $\mu_n\to \mu$ weakly. Since $\mu$ is supported by the finite set $\{X_1(\omega),\ldots,X_n(\omega)\}$, we are done.

Note that the result we used is maybe not the most simple way, and that pointwise convergence is not enough to conclude (since the almost everywhere depend on $t$).

-
@ Davide Giraudo, Thanks for you reply. But I didn't get the point of your last statement. What do you mean by saying that 'the almost everywhere depend on $t$'? – Julie Apr 13 '12 at 16:43
When you apply the LLN, you get for a fixed $t$ a set $N_t$ of measure $0$ such that $F_n(t,\omega)\to F(t)$ for $\omega\notin N_t$. But the union of $N_t$ for $t\in\mathbb R$ a priori may be not measurable or not of measure $0$. – Davide Giraudo Apr 13 '12 at 16:51

Sorry if this is not exactly an answer to your original question. But I get a little bit itchy when I see the way some probabilists treat weak convergence of measures...

The Riesz Representation Theorem states that when $X$ is locally compact Hausdorff, we have an isometry between the dual of $C_0(X)$ and the normed vector space of signed finite measures on the Borel sets of $X$. The norm of a measure is its total variation. In this context, weak convergence of measures is simply the weak-* convergence in $C_0(X)$.

The Banach-Alaoglu Theorem states that the closed unit ball is compact in the weak-* topology. It is very easy to show that the set of (positive) measures such that $0 \leq \mu(X) \leq 1$ is a closed (in the weak-* topology) subset of the unit ball, and therefore is also compact. Let's represent the set of those measures by $\mathcal{M}$. Notice that if $X$ is not compact, then the set of probability measures (on the Borel sets of $X$) is not compact in the weak-* topology. In fact, if we take a sequence $x_n \in X$ "convergin to $\infty$", then $$\frac{1}{n} \sum_{j=1}^n \delta_{x_n} \rightarrow 0.$$

Since $\mathcal{M}$ is compact and convex, the Krein-Milman Theorem says that $\mathcal{M}$ is the weak-* closure of the convex combination of extremal points of $\mathcal{M}$. The extremal points are points that are not non-trivial convex combination of other points of $\mathcal{M}$. Notice that these are the measure $0$, and the Dirac deltas $\delta_x$. Therefore, the convex combinations of those are the measures $\mu_n$ with finite support. Now, it only remains to show that we can use $\frac{1}{\mu_n(X)} \mu_n$ instead of $\mu_n$, to conclude that any probability measure is the weak-* limit of probability measures with finite support.

I realize that this is quite complicated. But IMHO, concepts borrowed from functional analysis should be regarded as such. Of course, this is a matter of taste, but the definition using $F_n(t)$ is too artificial EVEN when compared to the definition using the weak-* topology on $C_0(X)$.

-
One nit-pick, weak convergence of measures in probability is weak* convergence in $C_B (X)$ (bounded continuous functions) not $C_0 (X)$. Weak* convergence in $C_0(X)$ is what a probabilist calls "vague" convergence. – Chris Janjigian Apr 18 '12 at 13:08
@Chris: Thank you very much! I didn't know that. Do you have any references to this? – André Caldas Apr 18 '12 at 13:33
There is a proof in any graduate level textbook on probability that weak convergence in probability is equivalent to the statement that $Ef(X_n) \to Ef(X)$ for every bounded continuous function (this is the Portmanteau theorem). An easy counterexample to the statement that weak convergence is convergence in the weak* sense on $C_0 (X)$ is to consider Dirac masses going to infinity on the line, which will converge to zero vaguely but cannot converge weakly (in the probabilistic sense) because the zero measure is not a probability measure. – Chris Janjigian Apr 18 '12 at 14:07
@Chris: yes, this counter example is exactly the same as mine. This sequence has no limit in $C_B(X)^*$ with the weak-* topology because in this case, it would have to coincide with the limit in $C_0(X)^*$, but the limit in $C_B(X)^*$ is always a probability, because $1 \in C_B(X)$ implies that $1 = \mu_n(X) \rightarrow \mu(X)$. – André Caldas Apr 18 '12 at 14:17
@Chris: what I meant by "reference", was a reference on the differences between this "vague" convergence and the weak convergence, so I can fix my answer (and learn a bit). It seems to me that one advantage of the weak convergence is that convergent probabilities converge to a probability. But the "ball" is not compact. :-( – André Caldas Apr 18 '12 at 14:20