# Strong law of large numbers for transformations of non-iid random variables

Let $$X_1, X_2, \ldots$$ be a sequence of $$E$$-valued random variables (not necessarily iid), for $$(E,\mathcal{E})$$ some measurable space.

For any $$f : E\rightarrow\mathbb{R}$$ integrable, define $$\hat{\mathbb{E}}_n(f):=\frac{1}{n}\sum_{j=1}^n f(X_j)$$ and $$\mathbb{E}(f):=\mathbb{E}[f(X_1)]$$.

Suppose that there is an integrable $$f_0 : E\rightarrow\mathbb{R}$$ such that $$\tag{1} \hat{\mathbb{E}}_n(f_0)\longrightarrow \mathbb{E}(f_0) \quad \text{almost surely} \quad \text{as } \ n\rightarrow\infty.$$

Given regularity conditions of your choice on $$f_0$$, are you aware of 'minimal' dependence restrictions on the sequence $$(X_j)$$ that are sufficient to from (1) infer that $$\tag{2}\lim_{n\rightarrow\infty}\hat{\mathbb{E}}_n(f)=\mathbb{E}(f) \quad \text{a.s.} \quad \text{for \textbf{any} integrable } \ f:E\rightarrow\mathbb{R} \, ?$$

Edit: If $$f_0=\mathrm{id}_E$$, then $$(X_j)$$ iid guarantees ''$$(1)\Rightarrow (2)$$'' as desired. I'm interested in a choice of $$f_0$$ such that''$$(1)\Rightarrow (2)$$'' holds for a sequence $$(X_j)$$ with a less (least?) restrictive dependence structure.

• What is the role "integrability" of $f$ in your statement? So far, it didn't do anything even to make sure that $f(X_1)$ integrable. Commented Jun 2, 2021 at 17:11
• To be honest, you have pretty much nothing in your statement. <br> No relation between $f$ and $(X_n)$ ---> $( f(X_n))$ just can just be any real random variables <br> No useful constraint on $f_0$ just ---> it can be anything (For example how about constant function?) <br> No relation between $(X_n)$---> What to expect? Weak correlation? independence? Commented Jun 2, 2021 at 17:26
• I think you should be more specific on what you need or you should introduce more on your underlying problem unless you expect the answer in some form of "Wikipedia" Commented Jun 2, 2021 at 17:28
• @ParesseuxNguyen Thanks for your comments. As written in the above, the hypothesis of the question is that you are given a single 'germ' function $f_0$ (subject to conditions --- on just $f_0$ as well as compatibility conditions w.r.t. $(X_j)$ --- that are yet to be determined; hence the question) that should be 'structured enough' to guarantee the SLLN-convergence for all applicable $f$ as in (2). Commented Jun 2, 2021 at 23:16
• @ParesseuxNguyen Maybe the edit helped to clarify the question. Commented Jun 2, 2021 at 23:42

First of all, the condition that there exists $$f_0$$ is useless, there always exist function $$f_0(x)=0$$ which satisfy the condition.

Nevertheless, I may have something that may be interesting for you(or maybe not?). I will give two conditions that needs to be satisfied- one regards the "largeness" of each $$X_i$$, the second the "dependence between them".

Define $$Y_j=f(X_j)$$. Now, we only deal with real random variables and we want to know something about $$\frac{1}{n}\sum Y_j$$ (well, for every choice of $$f$$ right?). The most important condition is that expected value of $$Y_j$$ exists; $$\mathbb{E} |Y_j|<\infty$$. For example when you have the strong assumption that $$X_i$$ are iid, then if this is satisfied, then from LLN is really

$$\hat{\mathbb{E}}_n(f) = \frac{1}{n}\sum_{i=1}^n Y_i \to \mathbb{E}(Y_1)=\mathbb{E}(f) a.s.$$ On the other hand, if the finite expectation is NOT satisfied, then this does not hold even for iid random variables (again this followed from LLN). Therefore, we will assume that $$Y_i=f(X_i)$$ have always a finite expectation. Without that, we cannot expect that it will hold (this is the first "largeness"condition I was talking about)

Second, the "reasonable" condition that you are looking for is the ergodicity in the mean. There was a lot of research regarding the sufficient and necessary condition for the convergence of dependent sums, and Birkhoff–Khinchin theorem (or more generally the entire ergodic theory) gives you many sufficient conditions. For that I recommend google, it will give you better texts than I can write. Some examples of sufficiency include asymptotic independence of $$X$$ with $$X_i\overset{d}{=}X_j$$. But the "minimal conditions" (i.e. that are necessary AND sufficient) are very hard to give in the general case. n

Hope that helps

• Thanks for your answer, @Albert Paradek, though it's not quite what I was hoping for. Maybe my above comments help to clarify how this question was intended. Commented Jun 2, 2021 at 23:29