# Asymptotic distribution for MLE of exponential distribution

Let $X$ have an exponential distribution with parameter $\theta$ (pdf is $f(x, \theta) = \theta e^{-\theta x}$). I already found that the MLE for $\theta$ after $n$ observations is $$\hat{\theta}_{MLE} = \bar{X}^{-1} = \frac{n}{\sum_{i=1}^n{X_i}}$$ and that $\bar{X} \tilde{} \Gamma(n, n\theta)$.

The question is to derive directly (i.e. without using the general theory for asymptotic behaviour of MLEs) the asymptotic distribution of $$\sqrt n (\hat{\theta}_{MLE} - \theta)$$ as $n \to \infty$.

According to the general theory (which I should not be using), I am supposed to find that it is asymptotically $N(0, I(\theta)^{-1}) = N(0, \theta^2)$. However, I don't know where to start - for other distributions I was able to use CLT (if their MLE was the sample mean), but I can't think of a way to do it here.

• As a start, look up the inverse gamma distribution. – soakley Dec 31 '14 at 22:45

Since $\theta{}>{}0$, use the delta method . That is, given that, for $X \sim \mbox{exp}\left(\theta\right)$ i.i.d samples, the sample mean $\bar{X}_n$ is asymptotically normally distributed, so that $$\sqrt{n}\left(\bar{X}_n{}-{}\theta^{-1}\right){}\to{}N\left(0,\theta^{-2}\right)\,\mbox{as }n{}\to{}\infty\,.$$ Therefore, a first-order Taylor expansion of the function $\displaystyle \frac{1}{\bar{X}_n}$, in the "vicinity" of the asymptotic mean $\displaystyle \frac{1}{\theta}$, justifies
$$\sqrt{n}(\hat{\theta}_{MLE} - \theta){}={}\sqrt{n}(\frac{1}{\bar{X}_n} - \theta)\approx\theta^2\sqrt{n}\left(\bar{X}_n{}-{}\theta^{-1}\right){}\sim{}N\left(0,\theta^{2}\right)\,\mbox{, as }n{}\to{}\infty\,.$$ The "$\approx$" means that the random variables on either side have distributions that, with arbitrary precision, better approximate each other as $n{}\to{}\infty$. This approximation can be made rigorous.