Let $X_1,\dots,X_n$ be iid real valued random variables. Let $\mathcal{F}$ be a set of functions $f:\mathbb{R} \rightarrow \mathbb{R}$ such that $\mathbb{E}f(X_i) < \infty$ for all $f \in \mathcal{F}$.

For functions $l,u \in L^1(\mathbb{R})$, define $$ [l,u]:=\{f:\mathbb{R}\rightarrow\mathbb{R} : l(x)\leq f(x) \leq u(x) \text{ for all } x\in\mathbb{R}\}.$$ If $\mathbb{E}|u(X_i) - l(X_i)| \leq \epsilon$, such an object is called an $\epsilon$-bracket.

Suppose that for any $\epsilon>0$, there are finitely many $\epsilon$-brackets $[l_j,u_j]$, $j=1,\dots,N(\epsilon)$ such that for any $f\in \mathcal{F}$ there exists a $j$ with $f\in[l_j,u_j]$. Show that $$\underset{f\in\mathcal{F}}{\sup}\left|\frac{1}{n}\sum_{i=1}^nf(X_j) - \mathbb{E}f(X_j)\right|\rightarrow 0 \quad \text{a.s.}$$

I've been trying to understand how to approach this, but I'm completely stumped. I would like to say "let $f\in \mathcal{F}$ such that the supremum is obtained" and then work from there, but I'm not sure if that is a valid argument. Any ideas how to start this proof?


Fix $\varepsilon>0$ and let $l_j,u_j \in L^1$, $j=1,\ldots,N_{\varepsilon}$, such that for any $f \in \mathcal{F}$, there exists $j \in \{1,\ldots,N_{\varepsilon}\}$ such that $f \in [l_j,u_j]$. Since $\mathcal{G} := \{l_j,u_j; j=1,\ldots,N_{\varepsilon}\}$ is a finite set, there exists by the strong law of large numbers some $M=M(\varepsilon)$ such that

$$\sup_{g \in \mathcal{G}} \left| \frac{1}{n} \sum_{i=1}^n g(X_i) - \mathbb{E}g(X_1) \right| < \varepsilon \tag{1}$$

for all $n \geq M(\varepsilon)$.

Now let $f \in \mathcal{F}$ and $j$ such that $f \in [l_j,u_j]$. Then

$$\frac{1}{n} \sum_{i=1}^n l_j(X_i) - \mathbb{E}u_j(X_1) \leq \frac{1}{n} \sum_{i=1}^n f(X_i) - \mathbb{E}f(X_1) \leq \frac{1}{n} \sum_{i=1}^n u_j(X_i) - \mathbb{E}l_j(X_1).$$

Use the fact that $[l_j,u_j]$ is a $\varepsilon$-bracket, hence $\mathbb{E}|u_j(X_i)-l_j(X_i)| < \varepsilon$, in order to conclude that

$$-2\varepsilon \leq \frac{1}{n} \sum_{i=1}^n f(X_i) \leq 2 \varepsilon.$$

for all $n \geq M(\varepsilon)$. Since $\varepsilon>0$ is arbitrary and $M=M(\varepsilon)$ does not depend on $f$, this finishes the proof.

  • $\begingroup$ Thank you so much. Neat proof! $\endgroup$ – BallzofFury Sep 8 '14 at 12:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.