# What is the mle of $\Theta$ given a sample $X_{1},…,X_{n}$ (iid) with pdf $f(x;\Theta)=e^{-(x-\Theta)},\Theta\leq x<\infty,-\infty<\Theta<\infty$?

So this is what I have done:

$$L = \prod_{i=1}^{n} e^{-(x_{i}-\Theta)} H(x_{i}-\Theta)$$

$$L = e^{- \sum_{i=1}^{n} x_{i} + n\Theta} H(x_{(1)}-\Theta)$$

$$l = \ln(L) = -\sum_{i=1}^{n} x_{i} + n\Theta +\ln( H(x_{(1)}-\Theta) )$$

$$\frac{dl}{d\Theta} =n + \frac{ d (\ln( H(x_{(1)}-\Theta) )) }{d\Theta}$$

Where $H(n)$ is the unit step function and and $x_{(1)}$ is the 1st order statistic or the minimum of the sample. Continuing:

$$\frac{ d (\ln( H(x_{(1)}-\Theta) )) }{d\Theta} = \frac{1}{H(x_{(1)}-\Theta)} \delta (x_{(1)}-\Theta)(-1)$$

Where $\delta(n)$ is the delta function. Setting $\frac{dl}{d\Theta}=0$, we get:

$$n = \frac{1}{H(x_{(1)}-\Theta)} \delta (x_{(1)}-\Theta)$$

Expanding the delta function and then the step function, we get:

$$n = \frac{1}{H(x_{(1)}-\Theta)} \begin{pmatrix} 0 & \text{ if } x_{(1)}<\Theta \\ \infty & \text{ if } x_{(1)}=\Theta \\ 0 & \text{ if } x_{(1)}>\Theta \end{pmatrix} = \begin{pmatrix} \frac{0}{0} & \text{ if } x_{(1)}<\Theta \\ \infty & \text{ if } x_{(1)}=\Theta \\ 0 & \text{ if } x_{(1)}>\Theta \end{pmatrix}$$

We know that $n\geq 1$. So the only way this can be true is if $x_{(1)}=\Theta$. Thus, the mle of $\Theta$ must be $x_{(1)}$. I believe this answer is correct, and it makes intuitive sense. However, I just want to make sure that my reasoning and the way I did it is valid.

-

Firstly, when taking the logarithm $l = \log(L)$ you need to know that $L>0$. That is, you must assume $\Theta \leq x_{(1)}$. In order to make this step rigorous, you should remark that you are excluding the case $\Theta >x_{(1)}$ it is obvious that $L=0$ in this case, and so the maximum will not occur in this case.

From this point on, you may assume $H(x_{(1)}-\Theta)=1$.

As a side note, $H' = \delta$ is incorrect. This is a convenient abuse of notation to avoid introducing the (unnecessarily complex topic for this type of problem) of integration with respect to a discontinuous measure. The derivative of $H$ does not exist at $0$, so setting the derivative to 0 and solving this way makes no sense since $H'$ makes no sense at 0.

However, as I remarked earlier, we have that $H=1$ in the nontrivial case.

Thus we find that $l=-\sum x_i + n\Theta$ and we are trying to maximize on the interval $(-\infty,x_{(1)}]$. And we proceed to find the maximum occurs when $\Theta = x_{(1)}$ using standard mathods.

-
Thanks for your answer! I follow your argument. However, for $l=log(L)$ why doesn't $L \geq 0$ apply, since $log(0) = -\infty$? The answer may be unbounded but it is not like taking the log of a negative number which simply does not make sense (insofar as real numbers are concerned). –  kyphos Mar 21 '13 at 20:33
It is okay to consider $\log(0)=-\infty$ if you like, but we are trying to do calculus here. We would like to differentiate $\log(L)$, and if we are allowing it to take $-\infty$ we will run into an $\infty-\infty$ situation when trying to differentiate. If you are extra careful to keep track of where it happens then nothing bad will happen, but I find that it is best to avoid letting $\infty$ slip into a problem on $\mathbb{R}$ if it doesn't have to. –  nullUser Mar 22 '13 at 3:57
This makes sense. Thanks again! –  kyphos Mar 24 '13 at 2:40

You can not take the log likelihood since the likelihood is null for $\Theta \geq x_{(1)}$. Simply observe that $L$ is increasing for $\Theta \in (- \infty,x_{(1)})$ and is null for $\Theta \geq x_{(1)}$. So the maximum is at $x_{(1)}$.

-