# Approximation of sample mean distribution

Suppose that we have an iid sample $X_1,\dots,X_n$ with a distribution function $F$. Denote $\bar X_n:=\frac{1}{n}\sum_{i=1}^n X_i$ and $\bar X_n^*:=\frac{1}{n}\sum_{i=1}^n X_i^*,$ where $X_1^*,\dots,X_n^*$ are iid from the empirical distribution function $\hat F$ given the sample $X_1,\dots,X_n$. Does it then follow that $$\sup_x|\mathbb P(\sqrt{n}\bar X_n \leq x) - \mathbb P(\sqrt{n}\bar X_n^* \leq x)| \rightarrow 0 \ \ \mathrm{a.s.}$$

-
Shouldn't there be a limit $n\to \infty$ added? –  Raskolnikov Mar 6 '12 at 17:48

Suppose the variance of the distribution $F$ is $\sigma^2<\infty$, and denote $\mu$ the mean of $X_i$. By Lindeberg–Lévy CLT, we have $$\mathbb{P}\left(\sqrt{n}\bar{X}_n\leq x\right)\rightarrow_d N\left(\mu,\sigma^2\right).$$ Denote $\Phi_{\mu,\sigma}(x)$ the CDF of a normal distributed random variable with mean $\mu$ and variance $\sigma^2$ evaluated at $x$. Because $\Phi_{\mu,\sigma}(x)$ is continuous throughout $\mathbb{R}$, we further have $$\sup_x\left|\mathbb{{P}}\left(\sqrt{n}\bar{X}_{n}\leq x\right)-\Phi_{\mu,\sigma}\left(x\right)\right|\rightarrow0,\,n\rightarrow\infty.$$ Next, we can show that $$\mathbb{P}\left(\sqrt{n}\bar{X}_n^*\leq x\right)\rightarrow_d N\left(\mu,\sigma^2\right).$$ Note that $X_1^*,\ldots,X_n^*$ are iid, and $$E\left(X_{i}^{*}\right)=E\left\{ E\left(X_{i}^{*}\mid X_{1},\ldots,X_{n}\right)\right\} =E\left(\frac{1}{n}\sum_{i=1}^{n}X_{i}\right)=\mu.$$ Similaryly, we have $Var(X_i^*)=\sigma^2$. Then the above convergence follows from the CLT. And the convergence is also uniform. In the end, your claims holds because \begin{aligned} & \sup_{x}\left|\mathbb{P}\left(\sqrt{n}\bar{X}_{n}\leq x\right)-\mathbb{P}\left(\sqrt{n}\bar{X}_{n}^{*}\leq x\right)\right|\\ = & \sup_{x}\left|\mathbb{P}\left(\sqrt{n}\bar{X}_{n}\leq x\right)-\Phi_{\mu,\sigma}\left(x\right)+\Phi_{\mu,\sigma}\left(x\right)-\mathbb{P}\left(\sqrt{n}\bar{X}_{n}^{*}\leq x\right)\right|\\ \leq & \sup_{x}\left|\mathbb{P}\left(\sqrt{n}\bar{X}_{n}\leq x\right)-\Phi_{\mu,\sigma}\left(x\right)\right|+\sup_{x}\left|\mathbb{P}\left(\sqrt{n}\bar{X}_{n}^{*}\leq x\right)-\Phi_{\mu,\sigma}\left(x\right)\right|\\ = & 0. \end{aligned}