2
$\begingroup$

There is a classic problem:

Suppose that $X_1,\ldots,X_n$ form an i.i.d. sample from a distribution with the following pdf:

$$f(x\mid\theta) = \begin{cases} e^{\theta-x}\quad&\text{for }\, x> \theta \\ 0 &\text{otherwise}. \end{cases}$$

I would like to show that the MLE of $\theta$ does not exist.

The argument I have is that the likelihood function will be a maximum when $\theta$ is made as large as possible subject to the strict inequality $\theta < \min\{X_1, \ldots, X_n\}$. Therefore, the value $\theta = \min\{X_1,\ldots , X_n\}$ cannot be used and there is no MLE.

However, I do not understand WHY we want $\theta$ to the equal to the maximum of the values.

Also, is there a way to show mathematically why this MLE doesn't exist?

I get that the log-likelihood function is:

$$L(\theta) = n\theta - (X_1+\ldots+X_n)$$

but when you differentiate via $\theta$ and set to $0$, we get:

$n=0$. How does the fact $n=0$ fit into the fact the MLE doesn't exist for $\theta$? Thanks!

$\endgroup$
3
  • 1
    $\begingroup$ BTW: "n" is not a parameter in your formulation, it will be given. You will know your sample size, so solving wrt "n" doesn't mean anything. $\endgroup$
    – user76844
    Jan 29, 2014 at 17:29
  • $\begingroup$ however, maximizing $L(\theta)$ wrt $\theta$ does make sense. Note that your likelihood increases linearly for all $\theta\leq\min{x_i}$ but drops to $0$ when $\theta>\min{x_i}$, so your maximim likelihood estimate will be $\hat theta = \min{x_i}$ JPi and I have both given reasons for why the strict inequality in your formulation does not make sense. $\endgroup$
    – user76844
    Jan 29, 2014 at 17:34
  • $\begingroup$ math.stackexchange.com/q/289542/321264 $\endgroup$
    – StubbornAtom
    Mar 25, 2022 at 4:53

2 Answers 2

Reset to default
1
$\begingroup$

You have a continuous random variable so $\theta < \min{x_i} \equiv \theta \leq \min{x_i}\rightarrow \theta = \min{x_i}$

$\endgroup$
1
$\begingroup$

I don't know where you got this `classic' problem since it is incorrect. The density function you are describing is the same as

$$f(x|\theta) = \begin{cases} e^{\theta-x}, & x\geq \theta, \\ 0, & x<\theta.\end{cases}$$

because of the way that a density is defined as a Radon-Nikodym derivative, so the minimum of your observations is in fact the MLE.

$\endgroup$
5
  • $\begingroup$ If you notice, the x is STRICTLY greater than $theta$ in my example. Therefore, we CANNOT take the minimum. $\endgroup$
    – user123276
    Jan 26, 2014 at 8:06
  • $\begingroup$ Did you read my answer? Please read up on the formal definition of a density function. Wikipedia has a paragraph that should send you in the right direction, ie the notion of RN derivatives. $\endgroup$
    – JPi
    Jan 26, 2014 at 11:22
  • 2
    $\begingroup$ @user123276 a continuous random variable with the above density function will be defined at $x=\theta$, you cannot declare, by fiat, that x must be strictly greater than theta if x is a continuous random variable with a density defined on the points greater than theta. As an example, lets say that we calculate the probabilty of$x\geq \theta+\epsilon$ for $\epsilon > 0$, then $\lim_{\epsilon \rightarrow 0} P(x\geq \theta+\epsilon) = 1$, so the density will include $\theta$ in its domain so that you have a proper probability measure. $\endgroup$
    – user76844
    Jan 26, 2014 at 19:25
  • $\begingroup$ Right!!!!!!!!!! $\endgroup$
    – JPi
    Jan 26, 2014 at 20:01
  • $\begingroup$ is there anyway you can throw a figure on this it's hard for me to visualize $\endgroup$
    – O.rka
    Jan 24, 2016 at 21:22

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .