Let $X_1,\dots,X_n$ denote a sequence of dependent - let's say stationary and ergodic - sample from a distribution $F$, e.g. obtained as a stationary Markov chain. Then the empirical CDF is given by $$\hat F_n(z) = \frac{1}{n} \sum_{i=1}^n I(X_i \leq z).$$ The $X_i$ can be assumed to have all finite second and higher moments if necessary.

Now, I am interested in the convergence of $\hat F_n $ to $F$. In the independent case, there exists the classical Glivenko theorem which gives us uniform convergence. I have already found suitably generalized versions, which also hold for the case discussed here.

What I would like to know, is whether it also holds that $\hat F_n$ converges in $L^2$ to $F$, i.e., $$ \int_\mathbb{R} (\hat F_n(z) - F(z))^2 dz \rightarrow 0. $$

Can someone point point me towards some literature? I assume that this should be a standard textbook result, but coming from biology applications, I am not really familiar with the mathematical statistics literature.

Alternatively, can this be somehow proved from the Glivenko-Cantelli theorem?


1 Answer 1


I'll prove a stronger statement (assuming stationarity and ergodicity):

If $E[|X_1|]<\infty$, then $$ \int_{\mathbb R} \big|\hat F_n(z) - F(z)\big| dz\to 0,\quad n\to\infty, $$ almost surely.

Take some integer $N\ge 1$ and write $$ \int_{\mathbb R} \big|\hat F_n(z) - F(z)\big| dz\le \int_{-N}^N \big|\hat F_n(z) - F(z)\big| dz + \int_N^\infty \hat G_n(z)dz + \int_N^\infty G(z)dz, $$ where $\hat G_n(z) = F_n(-z) + (1-\hat F_n(z))$, $G(z) = F(-z) + (1- F(z))$. In view of the Glivenko-Cantelli theorem, $$ \limsup_{n\to\infty} \int_{\mathbb R} \big|\hat F_n(z) - F(z)\big| dz\le \limsup_{n\to\infty} \int_N^\infty \hat G_n(z)dz + \int_N^\infty G(z)dz $$ almost surely. Now by the ergodic theorem, $$ \int_N^\infty \hat G_n(z)dz = \frac1n\sum_{k=1}^n (|X_k|-N)\mathbf{1}_{|X_k|\ge N}\to E[(|X_1|-N)\mathbf{1}_{|X_1|\ge N}],\quad n\to\infty, $$ almost surely. Hence, $$ \limsup_{n\to\infty} \int_{\mathbb R} \big|\hat F_n(z) - F(z)\big| dz\le 2E[(|X_1|-N)\mathbf{1}_{|X_1|\ge N}] $$ almost surely for each $N\ge 1$. Letting $N\to\infty$, we get the claim.

Auxiliary identity For a non-negative random variable $Z$ $$ \int_N^\infty P(Z\ge z) dz = E[(Z-N])\mathbf{1}_{Z\ge N}]. $$ Proof $$ \int_N^\infty P(Z\ge z) dz = \int_N^\infty \int_z^\infty dF_Z(y) dz = \int_N^\infty\int_N^y dz\, dF_Z(y)\\ = \int_N^\infty (y-N) dF_Z(y) = E[(Z-N])\mathbf{1}_{Z\ge N}]. $$

Now observe that $P(|X_1|\ge x) = F(-z) + (1-F(z)) = G(z)$ for almost all $z$.

To prove a similar identity for empirical CDF, note that it is a CDF of a uniform distribution $Z_n$ on $\{X_1,\dots,X_n\}$, therefore $$ \int_N^\infty \hat G_n(z) dz = E[(|Z_n|-N)\mathbf{1}_{|Z_n|\ge N}] = \frac1n\sum_{k=1}^n (|X_k|-N)\mathbf{1}_{|X_k|\ge N}, $$ where the expectation is taken w.r.t. the distribution of $Z_n$ for fixed $X_1,\dots,X_n$.

  • $\begingroup$ Sorry, I was able to figure out the first question I had and deleted the comment, but the second question is still unclear to me. Or equivalently, why is $$ \int_N^\infty G(z)dz = E [(|X_1| - N)1_{|X_1|\geq N}]?$$ Sorry if these questions have obvious answers, I do lack some formal mathematics eduaction $\endgroup$ Commented Sep 25, 2015 at 13:03
  • $\begingroup$ @AuditorOfReality, I've updated the answer. Do ask if there is something still unclear. $\endgroup$
    – zhoraster
    Commented Sep 25, 2015 at 14:31

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .