Finding the maximum likelihood estimators for this shifted exponential PDF?

Question

Consider a random sample $X_1, X_2, \dots, X_n$ from the shifted exponential PDF

$$f(x; \lambda, \theta) = \begin{cases}\lambda e^{-\lambda(x-\theta)} ;& x \geq \theta\\ \theta ; &\text{Otherwise}\end{cases}$$

Taking $\theta = 0$ gives the pdf of the exponential distribution considered previously (with positive density to the right of zero).

a. Obtain the maximum likelihood estimators of $\theta$ and $\lambda$.

I followed the basic rules for the MLE and came up with:

$$\lambda = \frac{n}{\sum_{i=1}^n(x_i - \theta)}$$

Should I take $\theta$ out and write it as $-n\theta$ and find $\theta$ in terms of $\lambda$?

Answer 1

The density of a single observation $x_i$ is $$f(x \mid \lambda, \theta) = \lambda e^{-\lambda(x-\theta)} \mathbb{1}(x \ge \theta).$$ The joint density of the entire sample $\boldsymbol x$ is therefore $$\begin{align*} f(\boldsymbol x \mid \lambda, \theta) &= \prod_{i=1}^n f(x_i \mid \lambda, \theta) \\ &= \lambda^n \exp\left(-\sum_{i=1}^n \lambda(x_i - \theta)\right) \mathbb{1}(x_{(1)} \ge \theta) \\ &= \lambda^n \exp\left(-\lambda n (\bar x - \theta)\right) \mathbb{1}(x_{(1)} \ge \theta), \end{align*}$$ where $\bar x$ is the sample mean. Hence the joint log-likelihood for $\lambda, \theta$ is proportional to $$\ell(\lambda, \theta \mid \boldsymbol x) \propto \log \lambda - \lambda(\bar x - \theta) + \log \mathbb{1}(x_{(1)} \ge \theta).$$ The log-likelihood is maximized for a pair of estimators $(\hat \lambda, \hat \theta)$. Because $\lambda > 0$, $\ell$ is an increasing function of $\theta$ until $\theta > x_{(1)} = \min_i x_i$; hence $\ell$ is maximal with respect to $\theta$ when $\theta$ is made as large as possible without exceeding the minimum order statistic; i.e., $\hat \theta = x_{(1)}$. For a given $\theta$, $\ell$ with respect to $\lambda > 0$ is a continuous function, thus we compute the partial derivative $$\frac{\partial \ell}{\partial \lambda} = \frac{1}{\lambda} - (\bar x - \theta),$$ for which the only critical point is $$\lambda = \frac{1}{\bar x - \theta},$$ and we can verify that this choice is a global maximum for $\lambda > 0$. Therefore, our joint maximum likelihood estimator is $$(\hat \lambda, \hat \theta) = \left((\bar x - x_{(1)})^{-1}, x_{(1)}\right).$$ Note that when both $\lambda$ and $\theta$ are unknown parameters, the MLE cannot contain any expressions involving $\lambda$ or $\theta$, as an estimator is always a function of the sample and/or known parameters.

Answer 2

\begin{align} & L(\lambda,\theta) = \begin{cases} 0 \text{ if } \theta > \min\{x_1,\ldots,x_n\}, \text{ but otherwise as below:} \\[10pt] \displaystyle \lambda^n \exp\left(-\lambda\sum_{i=1}^n (x_i-\theta) \right) = \lambda^n\exp\left( -\lambda n (\overline x - \theta) \right) \text{ where } \overline x = \frac 1 n \sum_{i=1}^n x_i \end{cases} \end{align}

As a function of $\theta,$ this function increases as $\theta$ increases, until $\theta$ gets as big as $\min\{x_1,\ldots,x_n\}.$

Therefore the MLE for $\theta$ is $\min\{x_1,\ldots,x_n\}.$

Then we have $\displaystyle L(\lambda,\min) = \lambda^n \exp\left( -\lambda \sum_{i=1}^n (x_i-\min) \right),$ and so $$ \ell = \log L(\lambda,\min) = n\log\lambda - {}\lambda\sum_{i=1}^n(x_i-\min). $$ $$ \frac{d\ell}{d\lambda} = \frac n \lambda - \sum_{i=1}^n (x_i-\min). $$ This is $0$ when $\lambda = \dfrac n {\sum_{i=1}^n (x_i-\min)}.$

That doesn't prove that there is a global maximum at that point, but the nature of the function makes it clear that a global maximum occurs somewhere, and the derivative has to be $0$ where it occurs, and then we find that there is only one point where the derivative is $0.$ So that's it.