3
$\begingroup$

Studying for my upcoming exams I came upon this weird MLE exercise :

Let the random variable $X$ follow the Poisson Distribution with unknown parameter $\theta >0$. In $50$ observations of $X$, you only know that $20$ of them are zero. Find the Maximum Likelihood Estimator of $\theta$ using only this fact.

How would one proceed to finding the MLE using the fact that you have $20$ zero observations ? I have never came upon such a problem of MLE.

$\endgroup$

3 Answers 3

2
$\begingroup$

You have $X_1,\ldots,X_{50} \sim\operatorname{i.i.d. Poisson}(\lambda),$ so $\Pr(X_1=0) = e^{-\lambda}.$

The number of successes in $50$ trials with probability $e^{-\lambda}$ of success on each trial has a binomial distribution. So one has \begin{align} L(\lambda) & = \Pr(\text{exactly 20 such “successes'' in 50 trials)} \\[10pt] & = \binom{50}{20} \left( e^{-\lambda} \right)^{20} \left( 1 - e^{-\lambda}\right)^{30}. \end{align} We seek the value of $\lambda$ that minimizes that. Letting $p= e^{-\lambda},$ we can find the value of $p$ that minimizes the expression above, by writing \begin{align} \log L(\lambda) & = 20 \log p + 30\log(1-p) \\[10pt] \frac d{dp} \log L(\lambda) & = \frac{20} p - \frac{30}{1-p} = \frac{50((20/50) - p)}{p(1-p)} =\begin{cases} >0 & \text{if } 0 \le p < 20/50 \\ <0 & \text{if } 20/50 < p \le 1. \end{cases} \end{align}

$\endgroup$
1
  • $\begingroup$ why are we finding the value that minimizes it? Why not maximize , since we want to find MAXIMUM likelihood estimator $\endgroup$
    – Upstart
    Nov 4, 2022 at 5:34
2
$\begingroup$

Hints:

  • Given $\theta$, what is the probability of a zero in a particular observation? Of a non-zero?
  • Given $\theta$, what is the probability of $20$ zeros and $30$ non-zeros?
  • What is the likelihood proportional to?
  • Which $\theta$ maximises this likelihood?
$\endgroup$
2
$\begingroup$

Based on your description, your data can be summarized as $(X_i, \delta_i), i = 1, \dots, 50$, where \begin{align} \delta_i = \begin{cases} 1 & \text{ if } X_i = 0, \\ 0 & \text{ if } X_i > 0. \end{cases} \end{align}

If $\delta_i = 1$, then the contribution of the observation to the likelihood is $P(X_i = 0) = e^{-\theta} = e^{-\theta\delta_i}$. If $\delta_i = 0$, then the contribution of $X_i$ to the likelihood is $P(X_i > 0) = 1 - e^{-\theta} = (1 - e^{-\theta})^{1 - \delta_i}$. Each of these two cases may be written in a unified way as $(e^{-\theta})^{\delta_i}(1 - e^{-\theta})^{1 - \delta_i}$.

In summary, the likelihood function takes the form

$$L(\theta) = \prod_{i = 1}^{50} e^{-\theta\delta_i}(1 - e^{-\theta})^{1 - \delta_i}. $$

Or, equivalently, the log-likelihood $$\ell(\theta) = \sum_{i = 1}^{50}(-\theta \delta_i + (1 - \delta_i)\log(1 - e^{-\theta})) = -20\theta + 30\log(1 - e^{-\theta}), \tag{*}$$ where we used the condition $\sum_{i = 1}^{50}\delta_i = 20$.

You can then find the MLE $\hat{\theta}$ by maximizing $(*)$ with respect to $\theta$.

$\endgroup$
7
  • $\begingroup$ It's simpler to use $e^{-\theta}$ as the independent variable than to use $\theta$ itself. $\endgroup$ Jun 7, 2018 at 21:57
  • $\begingroup$ @MichaelHardy The OP asks to infer $\theta$ instead of $e^{-\theta}$. On the other hand, the simplicity you mentioned is merely marginal, if not more obscured. $\endgroup$
    – Zhanxiong
    Jun 7, 2018 at 21:59
  • $\begingroup$ But if you find the value $\widehat {\,p\,}$ of $p=e^{-\theta}$ that maximizes $L,$ then you can say say that the value of $\theta$ that maximizes $L$ is given by $\widehat {\,p\,} = e^{-\widehat{\,\theta\,}}$ and solving that for $\widehat{\,\theta\,}.$ and the differentiation is simpler, that simplicity being the reason for doing it that way. $\endgroup$ Jun 7, 2018 at 22:05
  • $\begingroup$ As i said, anyone who knows calculus would find such simplification is immaterial, while it brings unnecessary reparameterization. Anyway, it is fine to keep your opinion :) $\endgroup$
    – Zhanxiong
    Jun 7, 2018 at 22:13
  • 1
    $\begingroup$ To say "anyone who knows calculus" is wrong, since I know calculus very well. To minimize the distance $\sqrt{(x(t) - x_0)^2 + (y(t) - y_0)^2\,} $ it is simpler to directly minimize $(x(t) - x_0)^2 + (y(t) - y_0)^2,$ since the differentiation is simpler. Similarly, in Regiomontanus's angle maximization problem, instead of finding $d\theta/dx,$ it is simpler to find $d(\tan\theta)/dx,$ since the differentiation is less complicated. $\qquad$ $\endgroup$ Jun 7, 2018 at 22:26

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .