Let $f:\mathbb{R}^d\to\mathbb{R}$ be a continuous function and let $(X_i)_{i\in\mathbb{N}}$ be idependent and identical copies of the random variable $X$. Suppose all these random variabes are integrable. I know from the strong law of large numbers that $$ \lim_{n\to\infty}\frac{1}{n}\sum_{i=1}^n f(X_i) = E[f(X)] $$ with probability one. Suppose we have a sequence of continuous functions $(f_i)_{i\in\mathbb{N}}$ that converges to $f$ uniformly on compact sets. Does it hold that $$ \lim_{n\to\infty}\frac{1}{n}\sum_{i=1}^n f_n(X_i) = E[f(X)] $$ with probability one?

I know that if the convergence to $f$ is uniform, then the above is probably true. I'm curious about if it also holds for uniform convergence on compact sets.

Edit: My thoughts.

The following is to show that if the convergence to $f$ is uniform, then the above is true. Define $$ a_n := \sup_{x\in\mathbb{R}^n} |f(x) - f_n(x)|. $$ Observe that $$ \lim_{n\to\infty}\frac{1}{n}\sum_{i=1}^n f(X_i) - f_n(X_i) + f_n(X_i) = E[f(X)] $$ $$ \implies \lim_{n\to\infty}\frac{1}{n}\sum_{i=1}^n f_n(X_i) - \lim_{n\to\infty} \frac{1}{n}\sum_{i=1}^n f(X_i) + f_n(X_i) = E[f(X)]. $$ Therefore, if we have uniform convergence then $\lim_{n\to\infty} a_n = 0 \implies$ we have the desired result by using Cesaro means to show that the last sum on the left hand side of the equation goes to zero.

Does a similar or better argument exist for when there is uniform convergence on compact sets?


The result is false under the assumption of uniform convergence on compact sets.

Indeed, let the $X_i$ be i.i.d. exponential of rate 1, so $P(X_i>x) = e^{-x}$ for $x \geq 0$.

For $n \geq 1$, we define $f_n(x):= |x|$ for $|x| \leq \frac{1}{2}\log n$, we define $f_n(x) = 2n$ for $|x| \geq \log n$, and we define $f_n$ to be linearly interpolated in between those values. It is clear that the $f_n(x)$ converge to $|x|$ uniformly on compact sets. Moreover, we have that $$\sum_{n=1}^{\infty}P(f_n(X_n) = 2n) = \sum_{n=1}^{\infty} P(X_n\geq \log n) = \sum_{n=1}^{\infty} \frac{1}{n}=+\infty.$$In particular, the second Borel-Cantelli-lemma says that $f_n(X_n)=2n$ infinitely often, a.s. Therefore, $$\limsup_{n \to \infty} \frac{1}{n} \sum_{i=1}^n f_n(X_i) \geq 2 >1 = E|X|.$$Remark: In order for your statement to be true, you need a stronger convergence statement, for instance there exists some $\epsilon>0$ for which $\|(f_n -f)\cdot 1_{[-\epsilon n , \epsilon n]}\|_{\infty} \to 0$. Indeed, by the SLLN (or Borel-Cantelli), we know that $|X_n| \leq \epsilon n$ for all but finitely many $n$, a.s.

  • $\begingroup$ Thanks. I don't understand how it holds under the last condition. Can you elaborate on your remark, please? $\endgroup$ – master_goon Feb 21 '18 at 20:12
  • $\begingroup$ @master_goon because then for any $\delta>0$ it holds that $|f(X_n)-f_n(X_n)| \leq \delta$ for all but finitely many $n$. And we know that the limit of cesaro means does not depend on the omission of any finite number of terms. $\endgroup$ – shalop Feb 22 '18 at 1:52
  • $\begingroup$ So if my understanding is correct, one can replace $\|(f_n -f)\cdot 1_{[-\epsilon n , \epsilon n]}\|_{\infty} \to 0$ with $\|(f_n -f)\cdot 1_{\mathbf{B}_n}\|_{\infty} \to 0$ where $\mathbf{B}_n$ is an epsilon ball in $\mathbb{R}^d$ with radius $n$ centered at $0$? $\endgroup$ – master_goon Feb 23 '18 at 12:47
  • $\begingroup$ @master_goon you mean a ball of radius $\epsilon n$, which is true. $\endgroup$ – shalop Feb 23 '18 at 16:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.