4
$\begingroup$

Given i.i.d. random variables $\{X_n\}$, define the empirical cdf $$ \hat{F}_n(\omega, x) = \frac{1}{n} \sum_{i = 1}^n \mathbf{1}_{X_i(\omega) \leq x}$$ where $\omega \in \Omega$ and $x \in \mathbb{R}$. Show that $\hat{F}_n(\cdot, \cdot) \overset{d}{\to} F$ as $n \to \infty$, i.e. $$ P\left(\left\{\omega : \lim_{n \to \infty} \hat{F}_n(\omega, x) = F(x)\text{ for every } x \in C(F)\right\}\right)=1$$

I know that each $\hat{F}_n(\cdot, x)$ is a random variable, and each $\hat{F}_n(\omega, \cdot)$ is a cdf. Furthermore, for fixed $x \in \mathbb{R}$, we have $\hat{F}_n(\cdot, x) \overset{as}{\to} F$ by the Strong Law of Large Numbers; thus, each $$ P\left(\left\{\omega : \lim_{n \to \infty} \hat{F}_n(\omega, x) = F(x)\right\}\right) := P(A_x) = 1. $$

The problem is extending this to almost all $x$. If I only had to deal with countably many $x$, then I could just take a countable union of $A_x^c$ and obtain the result. It has been hinted that the problem can be reduced to this case, but I am still unable to figure out how.

$\endgroup$
1
  • $\begingroup$ can't you just approximate each $x$ by a countable sequence? $\endgroup$
    – lmaosome
    Apr 18, 2023 at 9:14

2 Answers 2

3
$\begingroup$

The following is a concise proof of Glivenko-Cantelli Theorem, it gives the uniform convergence, i.e., \begin{equation*} \mathsf{P}(\lim_{n\to\infty}\sup_t|F_n(t)-F(t)|=0)=1. \tag{1} \end{equation*} which also includes what you want.

By the strong law of Large numbers, as $n\to\infty$ \begin{gather*} F_n(t)=\frac1n\sum_{i=1}^{n}I_{\{X_i\le t\}} \stackrel{\text{a.s.}}{\longrightarrow}\mathsf{E}[I_{\{X_1\le t\}}]=F(t), \quad \forall t\in\mathbb{R}, \tag{2}\\ F_n(t-)=\frac1n\sum_{i=1}^{n}I_{\{X_i< t\}} \stackrel{\text{a.s.}}{\longrightarrow}\mathsf{E}[I_{\{X_1< t\}}]=F(t-), \quad \forall t\in\mathbb{R}. \tag{3} \end{gather*}

Given a fixed $\varepsilon>0$, take $$ t_0=-\infty, \quad t_i= \sup\{t: F(t)-F(t_{i-1})\le \varepsilon \}, \quad i\ge 1, $$ then there exist finite $k\le 1/\epsilon+1 $ and $-\infty=t_0<t_1<\cdots<t_k=\infty$ satisfy $$ F(t_i-)-F(t_{i-1})\le \epsilon,\qquad 1\le i \le k.$$ Now, for $t_{i-1}\le t<t_i$, \begin{gather*} F_n(t_{i-1})\le F_n(t)\le F_n(t_i-),\qquad F(t_{i-1})\le F(t)\le F(t_i-)\\ F_n(t)-F(t)\le F_n(t_i-) -F(t_i-)+\varepsilon,\\ F(t)-F_n(t)\le F(t_{i-1})- F_n(t_{i-1}) + \varepsilon. \end{gather*} Furthermore, \begin{align*} &\sup_{t\in\mathbb{R}}|F_n(t)-F(t)| \\ &\quad\le \sup_{1\le i\le k} [|F_n(t_{i-1})-F(t_{i-1})|+F_n(t_i-)-F(t_i-)] + \varepsilon \end{align*} Now from (2),(3) easy to get \begin{equation*} \mathsf{P}\Big(\varlimsup_{n\to\infty} \sup_{t\in\mathbb{R}}|F_n(t)-F(t)|\le \varepsilon\Big)=1. \end{equation*} Hence, (1) is also true.

$\endgroup$
3
  • $\begingroup$ This looks good, thanks (+1)! $\endgroup$ Apr 21, 2023 at 14:21
  • $\begingroup$ In order to construct such a partition you take $k \in \mathbb N$ in such a way that $\frac {1} {k + 1} \lt \varepsilon.$ Since $t_i$'s are jump points of jump $\geq \varepsilon$ and $F$ is incraesing it follows that $F(t_i) \geq i \varepsilon$ for any $i \geq 1.$ So if $F(t_i-) - F(t_{i-1}) \geq \varepsilon$ for some $1 \leq i \leq k,$ then $F(t_i) \geq (i + 1) \varepsilon$ and since $F$ has jump more than $\varepsilon$ at each $t_j,$ it then follows that $F(t_k) \geq (k + 1) \varepsilon \gt 1,$ a contradiction. Am I right? $\endgroup$
    – Anacardium
    Mar 26 at 7:45
  • $\begingroup$ @Anacardium Thank you for your commments. I add the details of how to get $\{t_i\}$. If you have any questions about it ,please tell me, I like discuss it. $\endgroup$
    – JGWang
    Mar 27 at 7:32
1
$\begingroup$

While the existing answer provides a valid route to proving the desired result via the Glivenko-Cantelli theorem, I would like to offer a more simpler approach that avoids some of the technicalities involved in the theorem's proof.

You have already shown (using Kolmogorov's SLLN) that given $x \in \mathbb R,$ there exists $A_x \subseteq \Omega$ with $\mathbb P \left (A_x \right ) = 1,$ such that for all $\omega \in A_x,$ $\lim\limits_{n \to \infty} \widehat {F_n} (\omega, x) = F(x).$ Let $A = \bigcap\limits_{x \in \mathbb Q} A_x.$ Since $\mathbb Q$ is countable, it follows that $\mathbb P (A) = 1.$ So $\lim\limits_{n \to \infty} \widehat {F_n} (\omega, x) = F(x),$ for all $\omega \in A$ and for all $x \in \mathbb Q.$ Now let $x_0 \in C(F)$ and choose $s,t \in \mathbb Q,$ such that $s \lt x_0 \lt t.$ Since $x \mapsto \widehat {F_n} (\omega, x)$ is a cdf for each $\omega \in \Omega,$ it follows that $\widehat {F_n} (\omega, s) \leq \widehat {F_n} (\omega, x_0) \leq \widehat {F_n} (\omega, t),$ for all $\omega \in A.$ Thus for all $\omega \in A$ we have $$F(s) = \lim\limits_{n \to \infty} \widehat {F_n} (\omega, s) \leq \liminf\limits_{n \to \infty} \widehat {F_n} (\omega, x_0) \leq \limsup\limits_{n \to \infty} \widehat {F_n} (\omega, x_0) \leq \lim\limits_{n \to \infty} \widehat {F_n} (\omega, t) = F(t).$$ This is true for any $s, t \in \mathbb Q$ with $s \lt x \lt t.$ Since $x_0 \in C(F),$ letting $s \to x_0^-$ and $t \to x_0^+,$ we have $$\lim\limits_{n \to \infty} \widehat {F_n} (\omega, x_0) = F(x_0)$$ for all $\omega \in A.$ This shows that $$A \subseteq \left \{\omega \in \Omega\ :\ \lim\limits_{n \to \infty} \widehat {F_n} (\omega, x) = 0\ \text {for all}\ x \in C(F) \right \}.$$ Since $\mathbb P (A) = 1,$ the required result follows. $\square$

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .