6
$\begingroup$

Let $X$ have an exponential distribution with parameter $\theta$ (pdf is $f(x, \theta) = \theta e^{-\theta x}$). I already found that the MLE for $\theta$ after $n$ observations is $$\hat{\theta}_{MLE} = \bar{X}^{-1} = \frac{n}{\sum_{i=1}^n{X_i}}$$ and that $\bar{X} \tilde{} \Gamma(n, n\theta)$.

The question is to derive directly (i.e. without using the general theory for asymptotic behaviour of MLEs) the asymptotic distribution of $$\sqrt n (\hat{\theta}_{MLE} - \theta)$$ as $n \to \infty$.

According to the general theory (which I should not be using), I am supposed to find that it is asymptotically $N(0, I(\theta)^{-1}) = N(0, \theta^2)$. However, I don't know where to start - for other distributions I was able to use CLT (if their MLE was the sample mean), but I can't think of a way to do it here.

$\endgroup$
  • $\begingroup$ As a start, look up the inverse gamma distribution. $\endgroup$ – soakley Dec 31 '14 at 22:45
6
$\begingroup$

Since $\theta{}>{}0$, use the delta method . That is, given that, for $X \sim \mbox{exp}\left(\theta\right)$ i.i.d samples, the sample mean $\bar{X}_n$ is asymptotically normally distributed, so that $$ \sqrt{n}\left(\bar{X}_n{}-{}\theta^{-1}\right){}\to{}N\left(0,\theta^{-2}\right)\,\mbox{as }n{}\to{}\infty\,. $$ Therefore, a first-order Taylor expansion of the function $\displaystyle \frac{1}{\bar{X}_n}$, in the "vicinity" of the asymptotic mean $\displaystyle \frac{1}{\theta}$, justifies

$$ \sqrt{n}(\hat{\theta}_{MLE} - \theta){}={}\sqrt{n}(\frac{1}{\bar{X}_n} - \theta)\approx\theta^2\sqrt{n}\left(\bar{X}_n{}-{}\theta^{-1}\right){}\sim{}N\left(0,\theta^{2}\right)\,\mbox{, as }n{}\to{}\infty\,. $$ The "$\approx$" means that the random variables on either side have distributions that, with arbitrary precision, better approximate each other as $n{}\to{}\infty$. This approximation can be made rigorous.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.