# Proving empirical distribution function is the effective estimator for cumulative distribution function

Let $$\hat{F}_{n}(x)$$ be the Empirical Distribution Function (EDF), where $$X_{1}...X_{n}$$ is an iid sample:

$$\hat{F}_n(x) = \frac{1}{n}\sum_{i=1}^n1_{\{X_i\leq x\}} \quad \forall x\in\mathbb{R}$$

It can be shown that $$\forall x$$ $$E\hat{F}_{n}(x)=F(x)$$, where $$F(x)$$ is a CDF. So the EDF is an unbiased estimator for CDF at any point.

Question: Can it be shown that it is also an "effective" estimator? Or what I mean (English isn't my 1st language so I'm afraid of using wrong terminology), for any other estimator $$G_{n}(x)$$ such that $$EG_{n}(x)=F(x)$$ we get $$Var(G_{n}(x))\ge Var(\hat{F}_{n}(x))$$

• Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking.
– Community Bot
Jan 4 at 20:58
• Given a CDF $F_X(x)$ and given $x\in\mathbb{R}$, define $\theta = F_X(x)$. You want an unbiased estimator of $\theta$ given $n$ i.i.d. observations $\{X_i\}$. This is related to the Cramer-Rao bound (CRB), see here: en.wikipedia.org/wiki/Cram%C3%A9r%E2%80%93Rao_bound Jan 4 at 21:47
• The correct terminology is 'minimum variance unbiased estimator'. See math.stackexchange.com/q/1140968/321264 Jan 6 at 8:36

The following shows that EDF is not always the min variance unbiased estimator, though it is optimal over a certain class of estimators, and also optimal when $$\{X_i\}$$ are Bernoulli.

Assume that $$\{X_i\}$$ are iid with CDF $$F_X(x)=P[X\leq x]$$, where $$X=X_1$$. Fix $$x \in \mathbb{R}$$. Define $$p=F_X(x)$$. We want an unbiased estimator of $$p$$ based on $$n$$ i.i.d. samples $$\{X_i\}_{i=1}^n$$.

To this end, define the "EDF" estimator: $$\hat{F}_n(x) = \frac{1}{n}\sum_{i=1}^n 1_{\{X_i\leq x\}}$$ where $$1_{\{X_i\leq x\}}$$ is an indicator function that is 1 if $$X_i\leq x$$ and $$0$$ else. Define $$Y_i=1_{\{X_i\leq x\}} \quad \forall i \in \{1, 2, 3, ...\}$$ Then $$\{Y_i\}$$ are i.i.d. $$Bern(p)$$ and $$\hat{F}_n(x)=\frac{1}{n}\sum_{i=1}^nY_i$$. It is clear that $$\hat{F}_n(x)$$ is an unbiased estimator of $$p$$ with variance $$p(1-p)/n$$.

Unbiased estimators based only on $$\{Y_i\}_{i=1}^n$$:

If we restrict attention only to estimators based on $$\{Y_1, ..., Y_n\}$$, rather than more general estimators based on $$\{X_1, ..., X_n\}$$, then it can be shown $$\frac{1}{n}\sum_{i=1}^nY_i$$ is the min variance unbiased estimator of $$p$$. This is because its variance meets the Cramer-Rao bound (CRB) with equality. See "Example: Single-Parameter Bernoulli experiment" here: https://en.wikipedia.org/wiki/Fisher_information

Counter-example for estimators based only on $$\{X_i\}_{i=1}^n$$:

We can get an "academic" counter-example by crafting a family of distributions parameterized by an unknown value $$q\in (0,1)$$.

Define a discrete random variable $$Z\in\{0,10\}$$ with $$P[Z=0]=q$$, $$P[Z=10]=1-q$$.

Define $$X=Z+q$$.

Let $$\{X_i\}_{i=1}^{\infty}$$ be i.i.d. with the same distribution as $$X$$. Fix $$x=5$$. We want to estimate $$p=P[X\leq 5]$$ based on $$n$$ observations of $$\{X_i\}$$. We have $$p=P[X\leq 5] = P[Z+q\leq 5] = P[Z=0] = q$$ The estimator $$\frac{1}{n}\sum_{i=1}^n 1_{\{X_i\leq 5\}}$$ is unbiased and has variance $$q(1-q)/n$$.

However, the estimator $$X_1-\lfloor X_1\rfloor$$ is unbiased and has variance 0.