Suppose $X_1, X_2, X_3 \stackrel{\text{i.i.d.}} \sim \operatorname{Exp}(\theta)$.

Exercise: derive the maximum likelihood estimator based on $X = (X_1, X_2, X_3)$.

What I've tried: the likelihood is given by $\prod\limits_{i = 1}^3 f(X_i\mid \theta) \, d\theta = \prod\limits_{i = 1}^3 \theta e^{-3\theta x} \, d\theta$. The log-likelihood is given by $\log L = 3\log\theta - 3\theta x \log(e) = 3\log\theta - 3\theta x.$ Take the derivative and set it equal to $0$ and I get $\hat{\theta} = \frac{1}{x}$.

My question:

  • How do I derive the maximum likelihood estimator based on $X = (X_1, X_2, X_3)$? I know my solution is probably not correct, but I don't know what else I should try.

You're really close.

Note that $$L(\theta)=\prod_{i=1}^{3}f(X_i \mid \theta)=\prod_{i=1}^{3}\theta e^{-3\theta X_i} = \theta^3 e^{-\theta \sum_{i=1}^{3}X_i}$$ The loglikelihood is given by $$\ell(\theta) = 3\ln \theta-\theta\sum_{i=1}^{3}X_i$$ whose derivative is $$\ell^{\prime}(\theta) = \dfrac{3}{\theta}-\sum_{i=1}^{3}X_i=0\implies\hat{\theta}=\dfrac{3}{\sum_{i=1}^{3}X_i}\text{.}$$ Note that you may need to show that $\hat{\theta}$ is indeed the value of $\theta$ which maximizes $\ell$; to do this, notice that $$\ell^{\prime\prime}(\theta)=\dfrac{-3}{\theta^2}<0$$ regardless of the value of $\theta$.


Two issues:

  • You've got $\displaystyle\prod_{i=1}^3 \big( \theta e^{-\theta x} \big)$ where you need $\displaystyle \prod_{i=1}^3 \big( \theta e^{-\theta x_i}\big).$

    Thus when you take the logarithm, you should get $3\log\theta - \theta\sum_{i=1}^3 x_i$ rather than $3\log\theta - 3\theta x.$

  • You should not have $d\theta$ unless you have some prior probability distribution on $\theta$ (in which case maximum likelihood is probably not what you would be doing).

The derivative of the logarithm with respect to $\theta$ is $$ \frac 3 \theta - \sum_{i=1}^3 x_i \quad \begin{cases} >0 & \text{if } 0<\theta< \frac 1 3 \sum_{i=1}^3 x_i, \\ \\ <0 & \text{if } \theta> \frac 1 3 \sum_{i=1}^3 x_i. \end{cases} $$ Merely finding that the derivative is $0$ at a certain point is not enough to prove there is a global maximum at that point, but the two inequalities above do it.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.