I am working on a problem. Show that for each probability measure $\mu$, there exists probability measure $\mu_n$ with finite support such that $\mu_n$ converges weakly to $\mu$. I am thinking about the empirical measure, which has the distribution function $F_n(t)=1/n\sum 1(X_i\le t)$. So from LLN, we have $F_n(t)\rightarrow F(t)$ for each fixed $t$ a.s. But the convergence here is almost surely. So does this still means $F_n$ converges to $F$ weakly?

  • $\begingroup$ Is $\mu$ defined on what kind of set? Is it defined over the reals, over some compact or locally compact topological space? $\endgroup$ – André Caldas Apr 17 '12 at 4:23

You got the idea. Let $F_n(t,\omega):=\frac 1n\sum_{j=1}^n\chi_{(-\infty,t]}(X_k(\omega))$, where $X_j$ are independent random variable of law $\mu$. By Glivenko-Cantelli theorem, also known as fundamental theorem of statistics, we know that for almost all $\omega\in\Omega$, we have $$\sup_{t\in\mathbb R}|F_n(t,\omega)-F(t)|\to 0.$$

Fix one of these $\omega$, and let $\mu_n$ the probability measure associated with the cumulative distribution function $F_n(t)=F_n(t,\omega)$. Since $F_n(t)\to F(t)$ at all points of continuity of $F$, we have that $\mu_n\to \mu$ weakly. Since $\mu$ is supported by the finite set $\{X_1(\omega),\ldots,X_n(\omega)\}$, we are done.

Note that the result we used is maybe not the most simple way, and that pointwise convergence is not enough to conclude (since the almost everywhere depend on $t$).

  • $\begingroup$ @ Davide Giraudo, Thanks for you reply. But I didn't get the point of your last statement. What do you mean by saying that 'the almost everywhere depend on $t$'? $\endgroup$ – Julie Apr 13 '12 at 16:43
  • $\begingroup$ When you apply the LLN, you get for a fixed $t$ a set $N_t$ of measure $0$ such that $F_n(t,\omega)\to F(t)$ for $\omega\notin N_t$. But the union of $N_t$ for $t\in\mathbb R$ a priori may be not measurable or not of measure $0$. $\endgroup$ – Davide Giraudo Apr 13 '12 at 16:51

Sorry if this is not exactly an answer to your original question. But I get a little bit itchy when I see the way some probabilists treat weak convergence of measures...

The Riesz Representation Theorem states that when $X$ is locally compact Hausdorff, we have an isometry between the dual of $C_0(X)$ and the normed vector space of signed finite regular measures on the Borel sets of $X$. The norm of a measure is its total variation. In this context, weak convergence of measures is simply the weak-* convergence in $C_0(X)$.

The Banach-Alaoglu Theorem states that the closed unit ball is compact in the weak-* topology. It is very easy to show that the set of (positive) measures such that $0 \leq \mu(X) \leq 1$ is a closed (in the weak-* topology) subset of the unit ball, and therefore is also compact. Let's represent the set of those measures by $\mathcal{M}$. Notice that if $X$ is not compact, then the set of probability measures (on the Borel sets of $X$) is not compact in the weak-* topology. In fact, if we take a sequence $x_n \in X$ "convergin to $\infty$", then $$ \frac{1}{n} \sum_{j=1}^n \delta_{x_n} \rightarrow 0. $$

Since $\mathcal{M}$ is compact and convex, the Krein-Milman Theorem says that $\mathcal{M}$ is the weak-* closure of the convex combination of extremal points of $\mathcal{M}$. The extremal points are points that are not non-trivial convex combination of other points of $\mathcal{M}$. Notice that these are the measure $0$, and the Dirac deltas $\delta_x$. Therefore, the convex combinations of those are the measures $\mu_n$ with finite support. Now, it only remains to show that we can use $\frac{1}{\mu_n(X)} \mu_n$ instead of $\mu_n$, to conclude that any probability measure is the weak-* limit of probability measures with finite support.

I realize that this is quite complicated. But IMHO, concepts borrowed from functional analysis should be regarded as such. Of course, this is a matter of taste, but the definition using $F_n(t)$ is too artificial EVEN when compared to the definition using the weak-* topology on $C_0(X)$.

  • $\begingroup$ One nit-pick, weak convergence of measures in probability is weak* convergence in $C_B (X)$ (bounded continuous functions) not $C_0 (X)$. Weak* convergence in $C_0(X)$ is what a probabilist calls "vague" convergence. $\endgroup$ – Chris Janjigian Apr 18 '12 at 13:08
  • $\begingroup$ @Chris: Thank you very much! I didn't know that. Do you have any references to this? $\endgroup$ – André Caldas Apr 18 '12 at 13:33
  • $\begingroup$ There is a proof in any graduate level textbook on probability that weak convergence in probability is equivalent to the statement that $Ef(X_n) \to Ef(X)$ for every bounded continuous function (this is the Portmanteau theorem). An easy counterexample to the statement that weak convergence is convergence in the weak* sense on $C_0 (X)$ is to consider Dirac masses going to infinity on the line, which will converge to zero vaguely but cannot converge weakly (in the probabilistic sense) because the zero measure is not a probability measure. $\endgroup$ – Chris Janjigian Apr 18 '12 at 14:07
  • $\begingroup$ @Chris: yes, this counter example is exactly the same as mine. This sequence has no limit in $C_B(X)^*$ with the weak-* topology because in this case, it would have to coincide with the limit in $C_0(X)^*$, but the limit in $C_B(X)^*$ is always a probability, because $1 \in C_B(X)$ implies that $1 = \mu_n(X) \rightarrow \mu(X)$. $\endgroup$ – André Caldas Apr 18 '12 at 14:17
  • $\begingroup$ @Chris: what I meant by "reference", was a reference on the differences between this "vague" convergence and the weak convergence, so I can fix my answer (and learn a bit). It seems to me that one advantage of the weak convergence is that convergent probabilities converge to a probability. But the "ball" is not compact. :-( $\endgroup$ – André Caldas Apr 18 '12 at 14:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.