# Empirical likelihood and distribution.

Reading Qin & Lawless (1994) we get the following:

Given the observations $\mathit{y}_1,\ldots,\mathit{y_n}$ of iid. random variables $\mathit{Y}_1,\ldots,\mathit{Y_n}$, the empirical likelihood is

$$\mathit{L}(\mathit{F}) = \prod_{\mathit{i} = 1}^{\mathit{n}} \mathit{f}(\mathit{y_i}),$$ where $\mathit{f}$ is the distribution of the random variables $\mathit{Y_i}$. Then this function is maximized by the empirical distribution function, which has the following form:

$$\mathit{F_n}(\mathit{y}) = \frac{1}{\mathit{n}} \left\lbrace \mathit{x} : \mathit{x} \in \lbrace {\mathit{x}_1,\ldots,\mathit{x_n}} \rbrace, \mathit{x} \leq \mathit{y} \right\rbrace = \frac{1}{\mathit{n}} \sum_{\mathit{i}=1}^{\mathit{n}} \mathit{I_i},$$

that is the number of observations in the sample which are less than or equal to $\mathit{y}$, divided by $\mathit{n}$ to get a distribution, and $\mathit{I_i}$ is the indicator function for $\mathit{x_i}\leq\mathit{y}$.

Why can they affirm this maximization so surely without further explanation? I am sure I am missing some key aspects.

Thank you!

• Maybe they have some appendix or special chapter where proofs are referred to to not break the flow? – mathreadler Apr 8 '17 at 15:25

So, $L(F)$ is not the product of distributions of r.v. $Y_i$. This is a product of probability mass functions $\mathbb P(Y=x_i)$. It is nonzero only if $\mathbb P(Y=x_i)>0$ for any $i=1,\ldots,n$. Next, the discrete distribution $F$ which takes values $x_1,\ldots,x_n$ only maximizes the product of p.m.f.'s. $$L(F)=\prod_{i=1}^n\mathbb P(Y=x_i)=\prod_{i=1}^n p_i.$$
Note that if all $x_i$ are distinct, the statement is well-known and obvious: maximum of $\prod_{i=1}^n\mathbb p_i$ with $p_1+\ldots+p_n=1$ is reached on the values $p_1=\ldots=p_n=\frac1n$. This is exactly the empirical distribution.
Less obvious is this fact for the case when $x_i$ are not distinct. Then $$L(F)=\prod_{i=1}^k p_i^{n_i},\quad p_1+\ldots+p_k=1, \quad n_1+\ldots+n_k=n,$$ where $k$ is the number of distinct values in the set $\{x_1,\ldots,x_n\}$ and $n_i$ is the number of repetitions of value $x_i$.
To find $p_1,\ldots,p_k$ that provide maximum, the method of Lagrange multipliers is useful. It give that maximum is reached for $p_i=\tfrac{n_i}{n}$. This is the empirical distribution too.