# Convergence properties of empirical CDF?

Let $X_1,\dots,X_n$ denote a sequence of dependent - let's say stationary and ergodic - sample from a distribution $F$, e.g. obtained as a stationary Markov chain. Then the empirical CDF is given by $$\hat F_n(z) = \frac{1}{n} \sum_{i=1}^n I(X_i \leq z).$$ The $X_i$ can be assumed to have all finite second and higher moments if necessary.

Now, I am interested in the convergence of $\hat F_n$ to $F$. In the independent case, there exists the classical Glivenko theorem which gives us uniform convergence. I have already found suitably generalized versions, which also hold for the case discussed here.

What I would like to know, is whether it also holds that $\hat F_n$ converges in $L^2$ to $F$, i.e., $$\int_\mathbb{R} (\hat F_n(z) - F(z))^2 dz \rightarrow 0.$$

Can someone point point me towards some literature? I assume that this should be a standard textbook result, but coming from biology applications, I am not really familiar with the mathematical statistics literature.

Alternatively, can this be somehow proved from the Glivenko-Cantelli theorem?

I'll prove a stronger statement (assuming stationarity and ergodicity):

If $E[|X_1|]<\infty$, then $$\int_{\mathbb R} \big|\hat F_n(z) - F(z)\big| dz\to 0,\quad n\to\infty,$$ almost surely.

Take some integer $N\ge 1$ and write $$\int_{\mathbb R} \big|\hat F_n(z) - F(z)\big| dz\le \int_{-N}^N \big|\hat F_n(z) - F(z)\big| dz + \int_N^\infty \hat G_n(z)dz + \int_N^\infty G(z)dz,$$ where $\hat G_n(z) = F_n(-z) + (1-\hat F_n(z))$, $G(z) = F(-z) + (1- F(z))$. In view of the Glivenko-Cantelli theorem, $$\limsup_{n\to\infty} \int_{\mathbb R} \big|\hat F_n(z) - F(z)\big| dz\le \limsup_{n\to\infty} \int_N^\infty \hat G_n(z)dz + \int_N^\infty G(z)dz$$ almost surely. Now by the ergodic theorem, $$\int_N^\infty \hat G_n(z)dz = \frac1n\sum_{k=1}^n (|X_k|-N)\mathbf{1}_{|X_k|\ge N}\to E[(|X_1|-N)\mathbf{1}_{|X_1|\ge N}],\quad n\to\infty,$$ almost surely. Hence, $$\limsup_{n\to\infty} \int_{\mathbb R} \big|\hat F_n(z) - F(z)\big| dz\le 2E[(|X_1|-N)\mathbf{1}_{|X_1|\ge N}]$$ almost surely for each $N\ge 1$. Letting $N\to\infty$, we get the claim.

Auxiliary identity For a non-negative random variable $Z$ $$\int_N^\infty P(Z\ge z) dz = E[(Z-N])\mathbf{1}_{Z\ge N}].$$ Proof $$\int_N^\infty P(Z\ge z) dz = \int_N^\infty \int_z^\infty dF_Z(y) dz = \int_N^\infty\int_N^y dz\, dF_Z(y)\\ = \int_N^\infty (y-N) dF_Z(y) = E[(Z-N])\mathbf{1}_{Z\ge N}].$$

Now observe that $P(|X_1|\ge x) = F(-z) + (1-F(z)) = G(z)$ for almost all $z$.

To prove a similar identity for empirical CDF, note that it is a CDF of a uniform distribution $Z_n$ on $\{X_1,\dots,X_n\}$, therefore $$\int_N^\infty \hat G_n(z) dz = E[(|Z_n|-N)\mathbf{1}_{|Z_n|\ge N}] = \frac1n\sum_{k=1}^n (|X_k|-N)\mathbf{1}_{|X_k|\ge N},$$ where the expectation is taken w.r.t. the distribution of $Z_n$ for fixed $X_1,\dots,X_n$.

• Sorry, I was able to figure out the first question I had and deleted the comment, but the second question is still unclear to me. Or equivalently, why is $$\int_N^\infty G(z)dz = E [(|X_1| - N)1_{|X_1|\geq N}]?$$ Sorry if these questions have obvious answers, I do lack some formal mathematics eduaction Commented Sep 25, 2015 at 13:03
• @AuditorOfReality, I've updated the answer. Do ask if there is something still unclear. Commented Sep 25, 2015 at 14:31