Suppose, $X_1, \dots, X_n$ are iid random vectors with some distribution $P$ on $\mathbb{R}^d$ and suppose $E_P(|X_1^k|)$ exists for all $k \leq K$ for some $K > 1$. Now, let $\hat P_n$ be the empirical distribution of $X$, constructed by putting weight $1/n$ on each observed $X_i$; let $F_n$ be its cdf.

If $\dim(X) = 1$, then by Glivenko-Cantelli, $\Vert F_n - F\Vert_\infty \rightarrow 0$ almost surely, and so, $E_{\hat P_n}(X^k) \rightarrow E_P(X^k)$, almost surely, for all $k < K$.

My question is, does this generalize to multiple dimensions? I know that the Glivenko-Cantelli result does not generalize directly without additional restrictions on $P$, but does almost sure convergence of moments hold for general $P$? If not, what kind of conditions are sufficient?

What about coordinate product moments? i.e. do we have $E_{\hat P_n}(X_{J}^kX_{J}^{k'}) \rightarrow E_{P}(X_{J}^kX_{J}^{k'})$ almost surely, where now $X_{J}$ represents the the sub-vector $(X^{(j)})_{j \in J}$ for some $J \subset \{1, \dots, \dim(X)\}$

EDIT: Does this simply follow from the Strong Law of Large numbers?

We should have $$E_{P_n}(X^k) = \sum_{i=1}^n X_i^k \xrightarrow{a.s.} E_{P}(X^k)~,$$ since $\{X_i\}_{i=1}^n$ is an iid sequence from $P$ and $E(|X^{k}|)< \infty$. Is this correct? I think this also directly generalizes to the coordinate product moments, no?

Changed title to reflect this.

  • 1
    $\begingroup$ Coincidentally, when I'm free, I see your posts. $\endgroup$ Nov 22, 2020 at 1:51

1 Answer 1


While the almost sure uniform convergence for empirical distribution functions is not clear.
The almost sure convergence for moments is pretty straightforward.

Let $\alpha$ be any multi-index in $\mathbb{N}^d$ such that $|\alpha|\le K$
You can see that:

$$\mathbb{E}_{\hat{P}_N}(X^{\alpha}) = \dfrac{1}{N}\left( \sum_{n=1}^N X^{\alpha}_n \right)$$

Clearly, $(X^{\alpha} ,n \ge 1)$ is a sequence of integrable real random variables.
Hence, by the strong law oflarge number, we have: $$ \dfrac{1}{N}\left( \sum_{n=1}^N X^{\alpha}_n \right) \longrightarrow \mathbb{E}_P(X^{\alpha}) \quad \text{a.e}$$

Hence, your conclusion. $\square$


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .