I'm confused on an application of the theorem stating that "almost sure convergence plus uniform integrability implies $L^1$ convergence".

The example that follows is from van der Vaart "Asymptotic Statistics" p.279 second paragraph of Section 19.4

Consider the random variables $X_1,...,X_n$ i.i.d, $X_i:\Omega\rightarrow \mathcal{X}$ with probability distribution $P$. Consider a random function $f_{\theta}(X_i)$ such that $f_\theta:\mathcal{X}\rightarrow \mathbb{R}$ and the index $\theta \in \Theta \subseteq \mathbb{R}^l$. Consider a class of functions $\mathcal{F}:=\{f_{\theta} \text{ s.t. } \theta \in \Theta\}$ which is $P$-Glivenko-Cantelli, i.e. $\sup_{f_{\theta}\in \mathcal{F}}|\frac{1}{n}\sum_{i=1}^nf_{\theta}(X_i)-\int_{\mathcal{X}}f_\theta(x)dP|\rightarrow_{a.s.}0$

Consider $\hat{\theta}_n\in \Theta$ where $\hat{\theta}_n:=g(X_1,...,X_n)$. Suppose exists $f_{\hat{\theta}_n}\in \mathcal{F}$ $\forall n$.

The author writes that if $f_{\hat{\theta}_n}$ converges almost surely to $f_{\theta_0}$ and the sequence is uniformly integrable we have that $\int_{\mathcal{X}} f_{\hat{\theta}_n}(x)dP\rightarrow_{a.s.}\int_{\mathcal{X}}f_{\theta_0}(x)dP$

Could you help me to understand the following points:

(1) Should $f_{\theta_0}\in \mathcal{F}$?

(2) Should $f_{\hat{\theta}_n}$ converges almost surely to $f_{\theta_0}$ be intended as $$f_{\hat{\theta}_n}(x)\rightarrow_{a.s.}f_{\theta_0}(x) \hspace{1cm} \forall x \in \mathcal{X}\hspace{1cm} (\star)$$ (i.e. we take the probability with respect to $\hat{\theta}_n$)?

(3) Similarly, the sequence is uniformly integrable should be intended as $$ \lim_{M\rightarrow \infty} \lim_{n\rightarrow \infty}E_{X_1,...,X_n}(|f_{\hat{\theta}_n}(x)|1(|f_{\hat{\theta}_n}(x)|>M))=0 \hspace{1cm} \forall x \in \mathcal{X}\hspace{1cm} (\star\star) $$ (i.e. we take the expectation with respect to $\hat{\theta}_n$)?

If (2) and (3) are correct, $(\star)$ and $(\star\star)$ imply $\lim_{n\rightarrow \infty}\int_{\mathcal{X}^n}f_{\hat{\theta}_n}(x)dP^n=\int_{\mathcal{X}^n}f_{\theta_0}(x)dP^n$ $\forall x \in \mathcal{X}$. How can I conclude $\int_{\mathcal{X}} f_{\hat{\theta}_n}(x)dP\rightarrow_{a.s.}\int_{\mathcal{X}}f_{\theta_0}(x)dP$?


I think the expectation (3) should be with respect to the space $X$ is defined on: $sup_n \int |f_{\theta_n}(x)|\{|f_{\theta_n}(x)|>M\}dP(x)<\epsilon$ for $M$ sufficiently large. (Separate issue: your definition of UI looks different from mine.) This relation holds almost surely wrt $\theta_n$, which is probably defined on the sequence space the $X_i$ are defined on, say $\Gamma$. So in fact the $M$ is random, $M(\gamma)$. So for a.e. $\gamma\in\Gamma$, $f_{\theta_n(\gamma)}\rightarrow f_{\theta_0(\gamma)}$, $f_{\theta_n(\gamma)}$ is UI, and $E_X[f_{\theta_n(\gamma)}(X)]\rightarrow E_X[f_{\theta_0(\gamma)}(X)]$.

  • $\begingroup$ Thanks. This is confusing me because $\{f_{\hat{\theta}_n}\}_n$ is a sequence of random functions thanks to the index $\hat{\theta}_n$ only. Hence, how can we define almost sure convergence or uniform integrability taking probabilities and expectations with respect to $X_i$ and holding $\hat{\theta}_n$ fixed? $\endgroup$
    – TEX
    Mar 2 '16 at 8:48
  • $\begingroup$ And in Lemma 19.24 the author states among the conditions $\int_{\mathcal{X}}(f_{\hat{\theta}_n}(x)-f_{\theta_0}(x))^2dP(x)$ converges in probability to $0$. In this case the convergence in probability is related to $\hat{\theta}_n$? $\endgroup$
    – TEX
    Mar 2 '16 at 17:33
  • $\begingroup$ @STF $\{f_{\hat{\theta}_n}\}_n$ is a random sequence with respect to the $\hat{\theta}_n$ (defined on $\Gamma$ in my answer, but the $\Omega$ in your question is what I should have used), ie a random map $\mathbb{Z}\rightarrow \mathcal{F}$. But each function in that sequence can be treated as random wrt to $X$ (not the sequence $X_i$) defined on a separate space from $\Gamma$. So you can integrate some random $f_{\hat{\theta}_n}$ wrt its $x$ argument. I agree its confusing, so take my interpretation with a grain of salt. $\endgroup$
    – snarfblaat
    Mar 2 '16 at 19:20
  • $\begingroup$ @STF Second question--yes. In that case it is clear, since the randomness of $f$ wrt to its $x$ argument is integrated out, the sequence of integrals is a sequence of random variables defined on the $\hat{\theta}_n$ space, i.e., the sequence space of the $X_i$. $\endgroup$
    – snarfblaat
    Mar 2 '16 at 19:29
  • $\begingroup$ Thanks. What is $X$ in your first comment? How it is different from $X_i$? Why you say $X_i$ is a sequence? $\endgroup$
    – TEX
    Mar 2 '16 at 19:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.