# Application of the theorem stating that "almost sure convergence plus uniform integrability implies $L^1$ convergence"

I'm confused on an application of the theorem stating that "almost sure convergence plus uniform integrability implies $L^1$ convergence".

The example that follows is from van der Vaart "Asymptotic Statistics" p.279 second paragraph of Section 19.4

Consider the random variables $X_1,...,X_n$ i.i.d, $X_i:\Omega\rightarrow \mathcal{X}$ with probability distribution $P$. Consider a random function $f_{\theta}(X_i)$ such that $f_\theta:\mathcal{X}\rightarrow \mathbb{R}$ and the index $\theta \in \Theta \subseteq \mathbb{R}^l$. Consider a class of functions $\mathcal{F}:=\{f_{\theta} \text{ s.t. } \theta \in \Theta\}$ which is $P$-Glivenko-Cantelli, i.e. $\sup_{f_{\theta}\in \mathcal{F}}|\frac{1}{n}\sum_{i=1}^nf_{\theta}(X_i)-\int_{\mathcal{X}}f_\theta(x)dP|\rightarrow_{a.s.}0$

Consider $\hat{\theta}_n\in \Theta$ where $\hat{\theta}_n:=g(X_1,...,X_n)$. Suppose exists $f_{\hat{\theta}_n}\in \mathcal{F}$ $\forall n$.

The author writes that if $f_{\hat{\theta}_n}$ converges almost surely to $f_{\theta_0}$ and the sequence is uniformly integrable we have that $\int_{\mathcal{X}} f_{\hat{\theta}_n}(x)dP\rightarrow_{a.s.}\int_{\mathcal{X}}f_{\theta_0}(x)dP$

Could you help me to understand the following points:

(1) Should $f_{\theta_0}\in \mathcal{F}$?

(2) Should $f_{\hat{\theta}_n}$ converges almost surely to $f_{\theta_0}$ be intended as $$f_{\hat{\theta}_n}(x)\rightarrow_{a.s.}f_{\theta_0}(x) \hspace{1cm} \forall x \in \mathcal{X}\hspace{1cm} (\star)$$ (i.e. we take the probability with respect to $\hat{\theta}_n$)?

(3) Similarly, the sequence is uniformly integrable should be intended as $$\lim_{M\rightarrow \infty} \lim_{n\rightarrow \infty}E_{X_1,...,X_n}(|f_{\hat{\theta}_n}(x)|1(|f_{\hat{\theta}_n}(x)|>M))=0 \hspace{1cm} \forall x \in \mathcal{X}\hspace{1cm} (\star\star)$$ (i.e. we take the expectation with respect to $\hat{\theta}_n$)?

If (2) and (3) are correct, $(\star)$ and $(\star\star)$ imply $\lim_{n\rightarrow \infty}\int_{\mathcal{X}^n}f_{\hat{\theta}_n}(x)dP^n=\int_{\mathcal{X}^n}f_{\theta_0}(x)dP^n$ $\forall x \in \mathcal{X}$. How can I conclude $\int_{\mathcal{X}} f_{\hat{\theta}_n}(x)dP\rightarrow_{a.s.}\int_{\mathcal{X}}f_{\theta_0}(x)dP$?

I think the expectation (3) should be with respect to the space $X$ is defined on: $sup_n \int |f_{\theta_n}(x)|\{|f_{\theta_n}(x)|>M\}dP(x)<\epsilon$ for $M$ sufficiently large. (Separate issue: your definition of UI looks different from mine.) This relation holds almost surely wrt $\theta_n$, which is probably defined on the sequence space the $X_i$ are defined on, say $\Gamma$. So in fact the $M$ is random, $M(\gamma)$. So for a.e. $\gamma\in\Gamma$, $f_{\theta_n(\gamma)}\rightarrow f_{\theta_0(\gamma)}$, $f_{\theta_n(\gamma)}$ is UI, and $E_X[f_{\theta_n(\gamma)}(X)]\rightarrow E_X[f_{\theta_0(\gamma)}(X)]$.

• Thanks. This is confusing me because $\{f_{\hat{\theta}_n}\}_n$ is a sequence of random functions thanks to the index $\hat{\theta}_n$ only. Hence, how can we define almost sure convergence or uniform integrability taking probabilities and expectations with respect to $X_i$ and holding $\hat{\theta}_n$ fixed?
– TEX
Mar 2, 2016 at 8:48
• And in Lemma 19.24 the author states among the conditions $\int_{\mathcal{X}}(f_{\hat{\theta}_n}(x)-f_{\theta_0}(x))^2dP(x)$ converges in probability to $0$. In this case the convergence in probability is related to $\hat{\theta}_n$?
– TEX
Mar 2, 2016 at 17:33
• @STF $\{f_{\hat{\theta}_n}\}_n$ is a random sequence with respect to the $\hat{\theta}_n$ (defined on $\Gamma$ in my answer, but the $\Omega$ in your question is what I should have used), ie a random map $\mathbb{Z}\rightarrow \mathcal{F}$. But each function in that sequence can be treated as random wrt to $X$ (not the sequence $X_i$) defined on a separate space from $\Gamma$. So you can integrate some random $f_{\hat{\theta}_n}$ wrt its $x$ argument. I agree its confusing, so take my interpretation with a grain of salt. Mar 2, 2016 at 19:20
• @STF Second question--yes. In that case it is clear, since the randomness of $f$ wrt to its $x$ argument is integrated out, the sequence of integrals is a sequence of random variables defined on the $\hat{\theta}_n$ space, i.e., the sequence space of the $X_i$. Mar 2, 2016 at 19:29
• Thanks. What is $X$ in your first comment? How it is different from $X_i$? Why you say $X_i$ is a sequence?
– TEX
Mar 2, 2016 at 19:31