0
$\begingroup$

Reading Qin & Lawless (1994) we get the following:

Given the observations $\mathit{y}_1,\ldots,\mathit{y_n}$ of iid. random variables $\mathit{Y}_1,\ldots,\mathit{Y_n}$, the empirical likelihood is

$$\mathit{L}(\mathit{F}) = \prod_{\mathit{i} = 1}^{\mathit{n}} \mathit{f}(\mathit{y_i}), $$ where $\mathit{f}$ is the distribution of the random variables $\mathit{Y_i}$. Then this function is maximized by the empirical distribution function, which has the following form:

$$ \mathit{F_n}(\mathit{y}) = \frac{1}{\mathit{n}} \left\lbrace \mathit{x} : \mathit{x} \in \lbrace {\mathit{x}_1,\ldots,\mathit{x_n}} \rbrace, \mathit{x} \leq \mathit{y} \right\rbrace = \frac{1}{\mathit{n}} \sum_{\mathit{i}=1}^{\mathit{n}} \mathit{I_i}, $$

that is the number of observations in the sample which are less than or equal to $\mathit{y}$, divided by $\mathit{n}$ to get a distribution, and $\mathit{I_i}$ is the indicator function for $\mathit{x_i}\leq\mathit{y}$.

Why can they affirm this maximization so surely without further explanation? I am sure I am missing some key aspects.

Thank you!

$\endgroup$
  • $\begingroup$ Maybe they have some appendix or special chapter where proofs are referred to to not break the flow? $\endgroup$ – mathreadler Apr 8 '17 at 15:25
1
$\begingroup$

Your citation is incorrect. The obvious result that the authors provide at the beginning of their paper looks as follows:

citation from p.302 at Qin & Lawless (1994)

So, $L(F)$ is not the product of distributions of r.v. $Y_i$. This is a product of probability mass functions $\mathbb P(Y=x_i)$. It is nonzero only if $\mathbb P(Y=x_i)>0$ for any $i=1,\ldots,n$. Next, the discrete distribution $F$ which takes values $x_1,\ldots,x_n$ only maximizes the product of p.m.f.'s. $$L(F)=\prod_{i=1}^n\mathbb P(Y=x_i)=\prod_{i=1}^n p_i.$$

Note that if all $x_i$ are distinct, the statement is well-known and obvious: maximum of $\prod_{i=1}^n\mathbb p_i$ with $p_1+\ldots+p_n=1$ is reached on the values $p_1=\ldots=p_n=\frac1n$. This is exactly the empirical distribution.

Less obvious is this fact for the case when $x_i$ are not distinct. Then $$ L(F)=\prod_{i=1}^k p_i^{n_i},\quad p_1+\ldots+p_k=1, \quad n_1+\ldots+n_k=n, $$ where $k$ is the number of distinct values in the set $\{x_1,\ldots,x_n\}$ and $n_i$ is the number of repetitions of value $x_i$.

To find $p_1,\ldots,p_k$ that provide maximum, the method of Lagrange multipliers is useful. It give that maximum is reached for $p_i=\tfrac{n_i}{n}$. This is the empirical distribution too.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.