# A law of large numbers for bounded processes

Let $$(\Omega,\mathcal F,P)$$ be a probability space and let $$([0,1],\mathcal B[0,1],\lambda$$) be the unit interval with Lebesgue measure on the Borel subsets of $$[0,1]$$. Let $$f:[0,1]\times \Omega \to \mathbb R$$ be $$\mathcal B[0,1]\otimes \mathcal F$$ measurable and bounded. I want to show the existence of a subsequence $$(n_k)$$ such that

$$\frac{1}{n_k}\sum_{i=1}^{n_k} f(t_i,\omega) \overset{L^1(\Omega,\mathcal F,P)}\to \int f(t,\omega)d\lambda \quad \text{as } k\to \infty$$

for $$\lambda^\infty$$-almost every sequence $$(t_i)\subset [0,1]$$, where $$\lambda^\infty$$ denotes the product measure on the $$\sigma$$-algebra $$\mathcal B^\infty:=\bigotimes_{n=1}^\infty \mathcal B[0,1]$$.

I tried the following:

Fix $$\omega\in\Omega$$. For each $$j\geq1$$, let $$((t_i),\omega)\mapsto f(t_j,\omega)$$ denote the composition of the maps $$((t_i),\omega)\mapsto (t_j,\omega)$$ and $$(t,\omega) \mapsto f(t,\omega)$$. Since the $$j$$th projection map $$(t_i)\mapsto t_j$$ on $$[0,1]^{\infty}:=\prod_{i=1}^{\infty} [0,1]$$ is $$\mathcal B^\infty$$ measurable, we see that $$f(t_j,\omega)$$ is $$\mathcal B^\infty\otimes \mathcal F$$ measurable. Moreover, by construction of the product measure $$\lambda^\infty$$, the projection maps are i.i.d. random variables on $$([0,1]^{\infty},\mathcal B^{\infty},\lambda^{\infty})$$, and so by the SLLN we get

$$\frac{1}{n}\sum_{i=1}^n f(t_i,\omega) \overset{\lambda^\infty\text{-a.s}}\to \int f(t,\omega)d\lambda \quad \text{as } n\to \infty$$

for each $$\omega\in \Omega$$. For each $$n\geq 1$$, define the map $$((t_i),\omega)\mapsto S^n((t_i),\omega):=\frac{1}{n}\sum_{i=1}^n f(t_i,\omega) -\int f(t,\omega)d\lambda.$$ By Fubini's theorem $$\omega\to \int f(t,\omega)d\lambda$$ is $$\mathcal F$$-measurable, and so $$S^n((t_i),\omega)$$ is seen to be $$\mathcal B^\infty\otimes \mathcal F$$ measurable. Also $$S^n((t_i),\omega)$$ is bounded since $$f$$ is bounded. Therefore the DCT gives

$$\int |S^n((t_i),\omega)| d\lambda^{\infty} \to 0 \quad \text{as } n\to \infty$$

for each $$\omega\in \Omega$$. By Fubini's theorem, these integrals are $$\mathcal F$$-measurable functions of $$\omega$$, and they are bounded, and so another application of the DCT gives

$$\int \int |S^n((t_i),\omega)| d\lambda^{\infty}dP \to 0 \quad \text{as } n\to \infty$$

We can interchange the order of integration by Fubini's theorem to obtain

$$\int \int |S^n((t_i),\omega)| dPd\lambda^{\infty} \to 0 \quad \text{as } n\to \infty$$

This says that $$\int |S^n((t_i),\omega)| dP \to 0$$ as $$n\to \infty$$ w.r.t. the $$L^1([0,1]^{\infty} ,\mathcal B^{\infty},\lambda^{\infty})$$ norm. We can extract a subsequence $$(n_k)$$ converging $$\lambda^{\infty}$$-almost surely, i.e. such that

$$\frac{1}{n_k}\sum_{i=1}^{n_k} f(t_i,\omega) \overset{L^1(\Omega,\mathcal F,P)}\to \int f(t,\omega)d\lambda \quad \text{as } k\to \infty$$ for $$\lambda^\infty$$-almost every sequence $$(t_i)\subset [0,1]$$.

Is this reasoning correct? Thanks a lot for your help.

• Let $f(t) = t$. Then $\int_0^1 f(t) dt = 1/2$. But if you fix a sequence $\{t_i\}$ such that $0\leq t_i\leq 1/4$ for all $i$, then any average of that sequence $\frac{1}{n_k}\sum_{i=1}^{n_k}f(t_i)$ will be at most $1/4$. Commented Jun 7, 2021 at 22:19
• @Michael Yes there exists sequences such that the result is false, but am trying to show that it holds for $\lambda^{\infty}$-almost every sequences. Commented Jun 7, 2021 at 23:18
• Looks good to me then. Here I am surprised you need a subsequence $n_k$, though I cannot see how to make it true more generally as $n\rightarrow \infty$. I see your last part brings this up using that if $E[|X_n|]\rightarrow 0$ then there is a subsequence $n_k$ such that $X_{n_k}\rightarrow 0$ almost surely. Commented Jun 8, 2021 at 16:14
• @Michael Yes exactly. It would be nice to avoid using a subsequence.... Commented Jun 8, 2021 at 16:30
• Good solution, though if my calculations are correct, this is still true with $n \rightarrow \infty$ in general not just with a subsequence. Commented Jun 8, 2021 at 23:16

First, I restate a lemma you might have seen multiple times.
Lemma (an LLN theorem) Let $$X_1,X_2,\dots$$ be a sequence of centered random variables such that they are uniformly bounded from below, pairwise uncorrelated and their variances are uniformly bounded, then almost surely $$\lim_{n \rightarrow \infty} \frac{X_1+X_2+\dots+X_n}{n} =0$$

Demonstration: See here $$\square$$.

Back to your question, we consider the sequence of random variables $$(X_n)$$ defined by $$X_n(\mathbf{t},\omega)= f(\mathbf{t}_n,\omega)-\int_{[0,1]}f(s,\omega)ds$$ in the probability space $$\left( [0,1]^{\mathbb{N}}\times \Omega \space\space,\space\space\mathcal{B}([0,1]^{\mathbb{N}} )\otimes \mathcal{F}\space\space,\space\space\lambda^{\infty}\otimes P\right)$$ ( Note that $$\mathbf{t} \in [0,1]^{\mathbb{N}}$$)
So according to our lemma, there is a set $$A \in \mathcal{B}([0,1]^{\mathbb{N}})\otimes \mathcal{F}$$ such that $$|A|=1$$ and for any $$(\mathbf{t},\omega) \in A$$, we have: $$\lim_{n \rightarrow \infty} \frac{f(\mathbf{t_1},\omega)+\dots+f(\mathbf{t}_n,\omega)}{n}= \int_{[0,1]}f(s,\omega)ds$$

Now, for any $$\mathbf{t} \in [0,1]^{\infty}$$, let $$B_\mathbf{t}=\left\{ \omega \in \Omega: \frac{f(\mathbf{t_1},\omega)+\dots+f(\mathbf{t}_n,\omega)}{n} \text{ doesn't converge to } \int_{[0,1]}f(s,\omega)ds \right\}$$ So, $$\int_{[0,1]^{\mathbb{N}}} P( B_{\mathbf{t}})d\lambda^{\infty}(\mathbf{t})=\int_{[0,1]^{\mathbb{N}}}\int_{\Omega} \mathbf{1}_{\omega \in B_{\mathbf{t}}}dP(\omega)d\lambda^{\infty}(\mathbf{t}) \stackrel{(*)}{\le} \int_{[0,1]^{\mathbb{N}}}\int_{\Omega} \mathbf{1}_{ (\mathbf{t},\omega) \not \in A}dP(\omega)d\lambda^{\infty}(\mathbf{t})=|A^c|=0$$ where in (*), we use the fact that for any pair $$(\mathbf{t},\omega)$$ if $$\omega \in B_{\mathbf{t}}$$ , $$(\mathbf{t},\omega)$$ can not be included in $$A$$ (definition of $$B$$ and property of $$A$$).

So $$P(B_\mathbf{t})=0$$ $$\lambda^{\infty}$$-almost surely.

Hence for $$\lambda^{\infty}$$-almost surely $$\mathbf{t} \in [0,1]^{\mathbb{N}}$$, $$\lim_{n \rightarrow \infty} \frac{f(\mathbf{t_1},\omega)+\dots+f(\mathbf{t}_n,\omega)}{n}= \int_{[0,1]}f(s,\omega)ds$$ $$P$$-almost surely.

Side note If I'm not wrong, all the needed measurability is well handled by the measurability of $$f$$. Side note 2 By $$|A|$$ and $$|A^c|$$ , I mean $$\lambda^{\infty}\otimes P( A)$$ and $$\lambda^{\infty}\otimes P( A^c)$$, respetively.

• The fact that the $X_n$ are uncorrelated is because of $\int X_n(\mathbf{t},\omega) d(\lambda^{\infty}\otimes P)=\int \int X_n(\mathbf{t},\omega) d\lambda^{\infty}d P=0$, $\int X_n(\mathbf{t},\omega) X_m(\mathbf{t},\omega) d(\lambda^{\infty}\otimes P)=\int \int X_n(\mathbf{t},\omega)X_m(\mathbf{t},\omega) d\lambda^{\infty}d P=\int \big[\int X_n(\mathbf{t},\omega)d\lambda^{\infty}\big] \big[\int X_m(\mathbf{t},\omega)d\lambda^{\infty}\big] d P=0$ using Fubini's theorem and the independence of the $\omega$ sections of the $X_n$ on $([0,1]^{\infty} ,\mathcal B^{\infty},\lambda^{\infty})$ right? Commented Jun 9, 2021 at 16:17
• I think you are right about the measurabillity note. My main concern was the $\mathcal B^{\infty}$ measurability of $\mathbf{t}\mapsto P( B_{\mathbf{t}})$, but this is a standard result of product measures since $B_{\mathbf{t}}$ is the $\mathbf{t}$ section of a $\mathcal B^\infty\otimes \mathcal F$ measurable set. Commented Jun 9, 2021 at 16:26
• Yeah. This is what I meant about the correlation and the measurability. Commented Jun 9, 2021 at 16:36
• Thanks a lot for your answer. This is a very nice result! Commented Jun 9, 2021 at 16:59