Let $X$ have an exponential distribution with parameter $\theta$ (pdf is $f(x, \theta) = \theta e^{-\theta x}$). I already found that the MLE for $\theta$ after $n$ observations is $$\hat{\theta}_{MLE} = \bar{X}^{-1} = \frac{n}{\sum_{i=1}^n{X_i}}$$ and that $\bar{X} \tilde{} \Gamma(n, n\theta)$.

The question is to derive directly (i.e. without using the general theory for asymptotic behaviour of MLEs) the asymptotic distribution of $$\sqrt n (\hat{\theta}_{MLE} - \theta)$$ as $n \to \infty$.

According to the general theory (which I should not be using), I am supposed to find that it is asymptotically $N(0, I(\theta)^{-1}) = N(0, \theta^2)$. However, I don't know where to start - for other distributions I was able to use CLT (if their MLE was the sample mean), but I can't think of a way to do it here.

  • $\begingroup$ As a start, look up the inverse gamma distribution. $\endgroup$
    – soakley
    Commented Dec 31, 2014 at 22:45
  • $\begingroup$ The Delta Method is probably an easier and more accurate way to solve this question. $\endgroup$
    – John Smith
    Commented May 6, 2022 at 12:42

1 Answer 1


Since $\theta{}>{}0$, use the delta method . That is, given that, for $X \sim \mbox{exp}\left(\theta\right)$ i.i.d samples, the sample mean $\bar{X}_n$ is asymptotically normally distributed, so that $$ \sqrt{n}\left(\bar{X}_n{}-{}\theta^{-1}\right){}\to{}N\left(0,\theta^{-2}\right)\,\mbox{as }n{}\to{}\infty\,. $$ Therefore, a first-order Taylor expansion of the function $\displaystyle \frac{1}{\bar{X}_n}$, in the "vicinity" of the asymptotic mean $\displaystyle \frac{1}{\theta}$, justifies

$$ \sqrt{n}(\hat{\theta}_{MLE} - \theta){}={}\sqrt{n}(\frac{1}{\bar{X}_n} - \theta)\approx\theta^2\sqrt{n}\left(\bar{X}_n{}-{}\theta^{-1}\right){}\sim{}N\left(0,\theta^{2}\right)\,\mbox{, as }n{}\to{}\infty\,. $$ The "$\approx$" means that the random variables on either side have distributions that, with arbitrary precision, better approximate each other as $n{}\to{}\infty$. This approximation can be made rigorous.


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .