# Why sample statistics converge to the right parameter

We know that for sample/empirical distribution function $F_n(x)$ we have that

a) $F_n(x)\xrightarrow[p]{}F(x)$ (pointwise convergence)

b) $\dfrac{\sqrt{n}(F_n(x)-F(x))}{\sqrt{F(x)(1-F(x))}}\xrightarrow[d]{}N(0,1)$

c) $F_n$ converges uniformly in probability to F.

My question is how do we prove that the sample moments of order $k$, and sample central moments of order $k$ converge to $E(X^k)$ and $E(X-E(X))^k$ respectively? (I think need to use the above empirical distribution function properties, but I do not know how, or which...)

Any help would be appreciated.

If you know how to explain that the sample statistics converge, without using any property of the empirical distribution, I would also be thankful.

• It doesn't just follow from the formula? – user14972 Aug 9 '14 at 18:54
• @Hurkyl what formula? From the empirical distribution definition? How do we do that? – An old man in the sea. Aug 9 '14 at 19:01
• At first glance, this looks like the sort of problem where you write down the formula relating the sample moment to the samples, work out the distribution of the sample moment, then use $F_n \to F$ to get your answer. But I could be wrong. – user14972 Aug 9 '14 at 19:05
• Ok. But how do I relate $1/n \sum X^k_i = g(F_n( \cdot))$, where $F_n(x)=\frac{\sum I[X_i\leq x]}{n}$ and g continuous? – An old man in the sea. Aug 9 '14 at 19:17
• My first thought is to write $\displaystyle\lim_{n\to\infty} \int_{-\infty}^\infty x^k \, dF_n(x)$. My second thought is that I could be lazy and instead of doing the whole thing from scratch I could look at Serfling's Limit Theorems of Mathematical Statistics and see if it's done there. ${}\qquad{}$ – Michael Hardy Aug 21 '14 at 12:57

I'm assuming that your sample consists of iid random variables $X_1,\dots, X_n$. The law of large numbers says that if a random variable $X$ has a finite expected value, then then the sample mean $\overline X=\frac1n\sum_{i=1}^n X_i$ converges to $E(X)$ in probability. Replacing $X$ by $X^k$, this implies that the sample moments of order $k$ converge in probability to $E(X^k)$, as long as $E(X^k)$ exists.
For the same reason, $\frac1n\sum_{i=1}^n (X_i-E(X))^k$ converges in probability to the central moment of order $k$ if it exists; however, to get convergence of the sample central moments, we need to prove a modification of this statement where $E(X)$ is replaced by $\overline X$. We can do this by expressing the $k$th sample central moment as a polynomial in terms of the $j$th sample moments, $j=1,\dots,k$ (e.g., see http://mathworld.wolfram.com/SampleCentralMoment.html), and then applying the convergence of sample moments.
If we have an infinite iid sequence $X_1,X_2,\dots$, and if we form a sequence of samples where the $n$th sample consists of the first $n$ variables in this sequence, $X_1,\dots,X_n$, then we can use the strong law of large numbers to strengthen the conclusions above: namely, we will get almost sure convergence (which is stronger than convergence in probability) of both the sample moments and sample central moments.