# A LLN type theorem on the supremum of functions of a RV

Let $X_1,\dots,X_n$ be iid real valued random variables. Let $\mathcal{F}$ be a set of functions $f:\mathbb{R} \rightarrow \mathbb{R}$ such that $\mathbb{E}f(X_i) < \infty$ for all $f \in \mathcal{F}$.

For functions $l,u \in L^1(\mathbb{R})$, define $$[l,u]:=\{f:\mathbb{R}\rightarrow\mathbb{R} : l(x)\leq f(x) \leq u(x) \text{ for all } x\in\mathbb{R}\}.$$ If $\mathbb{E}|u(X_i) - l(X_i)| \leq \epsilon$, such an object is called an $\epsilon$-bracket.

Suppose that for any $\epsilon>0$, there are finitely many $\epsilon$-brackets $[l_j,u_j]$, $j=1,\dots,N(\epsilon)$ such that for any $f\in \mathcal{F}$ there exists a $j$ with $f\in[l_j,u_j]$. Show that $$\underset{f\in\mathcal{F}}{\sup}\left|\frac{1}{n}\sum_{i=1}^nf(X_j) - \mathbb{E}f(X_j)\right|\rightarrow 0 \quad \text{a.s.}$$

I've been trying to understand how to approach this, but I'm completely stumped. I would like to say "let $f\in \mathcal{F}$ such that the supremum is obtained" and then work from there, but I'm not sure if that is a valid argument. Any ideas how to start this proof?

Fix $\varepsilon>0$ and let $l_j,u_j \in L^1$, $j=1,\ldots,N_{\varepsilon}$, such that for any $f \in \mathcal{F}$, there exists $j \in \{1,\ldots,N_{\varepsilon}\}$ such that $f \in [l_j,u_j]$. Since $\mathcal{G} := \{l_j,u_j; j=1,\ldots,N_{\varepsilon}\}$ is a finite set, there exists by the strong law of large numbers some $M=M(\varepsilon)$ such that

$$\sup_{g \in \mathcal{G}} \left| \frac{1}{n} \sum_{i=1}^n g(X_i) - \mathbb{E}g(X_1) \right| < \varepsilon \tag{1}$$

for all $n \geq M(\varepsilon)$.

Now let $f \in \mathcal{F}$ and $j$ such that $f \in [l_j,u_j]$. Then

$$\frac{1}{n} \sum_{i=1}^n l_j(X_i) - \mathbb{E}u_j(X_1) \leq \frac{1}{n} \sum_{i=1}^n f(X_i) - \mathbb{E}f(X_1) \leq \frac{1}{n} \sum_{i=1}^n u_j(X_i) - \mathbb{E}l_j(X_1).$$

Use the fact that $[l_j,u_j]$ is a $\varepsilon$-bracket, hence $\mathbb{E}|u_j(X_i)-l_j(X_i)| < \varepsilon$, in order to conclude that

$$-2\varepsilon \leq \frac{1}{n} \sum_{i=1}^n f(X_i) \leq 2 \varepsilon.$$

for all $n \geq M(\varepsilon)$. Since $\varepsilon>0$ is arbitrary and $M=M(\varepsilon)$ does not depend on $f$, this finishes the proof.

• Thank you so much. Neat proof! – BallzofFury Sep 8 '14 at 12:47