Let $X_1,X_2,X_3....X_n$ be a random sample from a population having probability density function
$$f(x)=\dfrac{1}{\theta} e^\frac{-(x-\theta)}{\theta} ;x\geq\theta$$

where $\theta\in(0,\infty)$. Find the MLE of $\theta$.

In this question when i followed normal method of finding MLE(taking log likelihood differentiating with respect to $\theta$)I found $\hat\theta =\bar x$. But like in uniform distribution case we have here parameter dependent on $x$ . Now i am not sure if MLE is $\bar x$ or $x_{(1)}$.

  • 2
    $\begingroup$ Any maximum likelihood estimator cannot be greater than $x_{(1)}$ here as that would lead to a likelihood of $0$. It might be clearer if you wrote $f(x)=\dfrac{1}{\theta} e^\frac{-(x-\theta)}{\theta} I_{x \ge \theta}$ for all $x$ where $I_{x \ge \theta}$ is an indicator function $\endgroup$ – Henry Feb 1 '18 at 16:16
  • $\begingroup$ Likelihood in this $0 \leq \theta \leq x_{(1)}$ part will still give $\bar x $ i don't get it . $\endgroup$ – Daman Feb 1 '18 at 16:37
  • $\begingroup$ @Damn1o1 No, because $x_{(1)}$ is always less than or equal to $\bar x$ $\endgroup$ – heropup Feb 1 '18 at 16:40
  • $\begingroup$ @Henry After writing first comment you never comeback. $\endgroup$ – Daman Feb 1 '18 at 16:56
  • $\begingroup$ Damn1o1 - I am not permanently on the site. $\hat{\theta} = \bar x$ will have a zero likelihood because it is almost surely greater than $x_{(1)}$. @heropup gave the details $\endgroup$ – Henry Feb 1 '18 at 17:39

Note that $$f(x) = \frac{1}{\theta} \exp\left( - \left(\tfrac{x}{\theta} - 1 \right) \right) \mathbb 1 (x \ge \theta > 0) = \frac{e}{\theta} e^{-x/\theta} \mathbb 1 (x \ge \theta > 0).$$ Hence the likelihood given a sample $\boldsymbol x$ is proportional to $$\mathcal L(\theta \mid \boldsymbol x) = \theta^{-n} e^{-n \bar x/\theta} \mathbb 1(x_{(1)} \ge \theta > 0).$$ The log-likelihood is $$\ell(\theta \mid \boldsymbol x) = \begin{cases} - n \log \theta - \frac{n \bar x}{\theta}, & 0 < \theta \le x_{(1)} \\ -\infty, & \text{otherwise.}\end{cases}$$ But since the critical point satisfying $$\frac{\partial \ell}{\partial \theta} = 0 = -\frac{n}{\theta} \left( 1 - \frac{\bar x}{\theta}\right)$$ is $\theta = \bar x$, and $$\bar x = \frac{1}{n} \sum_{i=1}^n x_i \ge \frac{1}{n} \sum_{i=1}^n x_{(1)} = x_{(1)},$$ with equality if and only if $x_i = x_{(1)}$ for all $i = 1, \ldots, n$, we conclude that the log-likelihood has no critical point in $\theta \in (0, x_{(1)})$; hence it is maximized when $\theta = x_{(1)}$ as $\partial \ell/\partial \theta < 0$ for all such $\theta$ (i.e., it is strictly decreasing).

  • $\begingroup$ Thank you so much i get it now. $\endgroup$ – Daman Feb 1 '18 at 16:43
  • $\begingroup$ So $\bar x $ is nothing but $x_{(1)}$ because there nothing else except $x_{(1)}$ according to $\theta \in (0, x_{(1)})$. I am thinking it this way did i follow it right? $\endgroup$ – Daman Feb 1 '18 at 16:48
  • 2
    $\begingroup$ Not really. Unless all of your observations are the same (which for such a distribution happens with zero probability for a sample size greater than 1), the sample mean is always greater than the minimum order statistic, hence it cannot be chosen as the MLE. The likelihood is zero that $\theta$ is greater than the minimum order statistic, because you cannot choose the parameter to exceed a value that you observed. $\endgroup$ – heropup Feb 1 '18 at 16:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.