Weak convergence of probability measure I am working on a problem. Show that for each probability measure $\mu$, there exists probability measure $\mu_n$ with finite support such that $\mu_n$ converges weakly to $\mu$. I am thinking about the empirical measure, which has the distribution function $F_n(t)=1/n\sum 1(X_i\le t)$. So from LLN, we have $F_n(t)\rightarrow F(t)$ for each fixed $t$ a.s. But the convergence here is almost surely. So does this still means $F_n$ converges to $F$ weakly?
 A: Sorry if this is not exactly an answer to your original question.
But I get a little bit itchy when I see the way some probabilists treat
weak convergence of measures...
The Riesz Representation Theorem states that when $X$
is locally compact Hausdorff, we have an isometry between
the dual of $C_0(X)$ and the normed vector space of signed finite
regular measures on the Borel sets of $X$.
The norm of a measure is its total variation.
In this context, weak convergence of measures is simply the
weak-* convergence in $C_0(X)$.
The Banach-Alaoglu Theorem states that the closed unit ball
is compact in the weak-* topology.
It is very easy to show that the set of (positive) measures such that
$0 \leq \mu(X) \leq 1$ is a closed (in the weak-* topology) subset
of the unit ball, and therefore is also compact.
Let's represent the set of those measures by $\mathcal{M}$.
Notice that if $X$ is not compact, then the set of probability measures
(on the Borel sets of $X$) is not compact in the weak-* topology.
In fact, if we take a sequence $x_n \in X$ "convergin to $\infty$",
then
$$
  \frac{1}{n} \sum_{j=1}^n \delta_{x_n}
  \rightarrow
  0.
$$
Since $\mathcal{M}$ is compact and convex, the
Krein-Milman Theorem
says that $\mathcal{M}$ is the weak-* closure of the
convex combination of extremal points of $\mathcal{M}$.
The extremal points are points that are not non-trivial
convex combination of other points of $\mathcal{M}$.
Notice that these are the measure $0$, and the
Dirac deltas $\delta_x$.
Therefore, the convex combinations of those are the
measures $\mu_n$ with finite support.
Now, it only remains to show that we can use
$\frac{1}{\mu_n(X)} \mu_n$ instead of $\mu_n$, to conclude
that any probability measure is the weak-*
limit of probability measures with finite support.
I realize that this is quite complicated.
But IMHO, concepts borrowed from functional analysis
should be regarded as such.
Of course, this is a matter of taste, but the definition
using $F_n(t)$ is too artificial EVEN when compared to
the definition using the weak-* topology on $C_0(X)$.
A: You got the idea. Let $F_n(t,\omega):=\frac 1n\sum_{j=1}^n\chi_{(-\infty,t]}(X_k(\omega))$, where $X_j$ are independent random variable of law $\mu$. By Glivenko-Cantelli theorem, also known as fundamental theorem of statistics, we know that for almost all $\omega\in\Omega$, we have
$$\sup_{t\in\mathbb R}|F_n(t,\omega)-F(t)|\to 0.$$
Fix one of these $\omega$, and let $\mu_n$ the probability measure associated with the cumulative distribution function $F_n(t)=F_n(t,\omega)$. Since $F_n(t)\to F(t)$ at all points of continuity of $F$, we have that $\mu_n\to \mu$ weakly. Since $\mu$ is supported by the finite set $\{X_1(\omega),\ldots,X_n(\omega)\}$, we are done.
Note that the result we used is maybe not the most simple way, and that pointwise convergence is not enough to conclude (since the almost everywhere depend on $t$).
