# Asymptotic distribution for MLE of exponential distribution

Let $X$ have an exponential distribution with parameter $\theta$ (pdf is $f(x, \theta) = \theta e^{-\theta x}$). I already found that the MLE for $\theta$ after $n$ observations is $$\hat{\theta}_{MLE} = \bar{X}^{-1} = \frac{n}{\sum_{i=1}^n{X_i}}$$ and that $\bar{X} \tilde{} \Gamma(n, n\theta)$.

The question is to derive directly (i.e. without using the general theory for asymptotic behaviour of MLEs) the asymptotic distribution of $$\sqrt n (\hat{\theta}_{MLE} - \theta)$$ as $n \to \infty$.

According to the general theory (which I should not be using), I am supposed to find that it is asymptotically $N(0, I(\theta)^{-1}) = N(0, \theta^2)$. However, I don't know where to start - for other distributions I was able to use CLT (if their MLE was the sample mean), but I can't think of a way to do it here.

• As a start, look up the inverse gamma distribution. Commented Dec 31, 2014 at 22:45
• The Delta Method is probably an easier and more accurate way to solve this question. Commented May 6, 2022 at 12:42

Since $\theta{}>{}0$, use the delta method . That is, given that, for $X \sim \mbox{exp}\left(\theta\right)$ i.i.d samples, the sample mean $\bar{X}_n$ is asymptotically normally distributed, so that $$\sqrt{n}\left(\bar{X}_n{}-{}\theta^{-1}\right){}\to{}N\left(0,\theta^{-2}\right)\,\mbox{as }n{}\to{}\infty\,.$$ Therefore, a first-order Taylor expansion of the function $\displaystyle \frac{1}{\bar{X}_n}$, in the "vicinity" of the asymptotic mean $\displaystyle \frac{1}{\theta}$, justifies
$$\sqrt{n}(\hat{\theta}_{MLE} - \theta){}={}\sqrt{n}(\frac{1}{\bar{X}_n} - \theta)\approx\theta^2\sqrt{n}\left(\bar{X}_n{}-{}\theta^{-1}\right){}\sim{}N\left(0,\theta^{2}\right)\,\mbox{, as }n{}\to{}\infty\,.$$ The "$\approx$" means that the random variables on either side have distributions that, with arbitrary precision, better approximate each other as $n{}\to{}\infty$. This approximation can be made rigorous.