# Asymptotic Variance of MLE Exponential

Suppose we have a random sample $$(X_{1},.....,X_{n})$$, where $$X_{i}$$ follows an Exponential Distribution with parameter $$\lambda$$, hence:

$$F(x) = 1 - exp(-\lambda x)$$

$$\mathbb{E}(X_{i}) = \dfrac{1}{\lambda}$$

$$Var(X_{i}) =\dfrac{1}{\lambda^{2}}$$

I know that the MLE estimator $$\hat{\lambda} = \dfrac{n}{\sum_{i=1}^{n}X_{i}}$$, asymptotically follows a normal distribution, but I'm interested in his variance. So, since $$\sqrt n (\hat{\lambda} - \lambda)\stackrel{D}{\rightarrow} \mathcal{N}(0, \sigma^{2})$$

I was thinking about dealing with:

\begin{align} \sigma^{2} &= Var\left[\sqrt n (\hat{\lambda} - \lambda)\right]\\ &= n Var (\hat{\lambda})\\ &= n Var \left[ \dfrac{n}{\sum_{i=1}^{n}X_{i}} \right] \\ &= n^{3} Var \left[ \dfrac{1}{\sum_{i=1}^{n}X_{i}}\right] \end{align}

But I don't know how to proceed from here. I think the sum of exponentials follows a gamma with parameters $$(n,\lambda)$$, and that $$\dfrac{1}{\sum_{i=1}^{n}X_{i}}$$ then follows an inverse gamma?

Yes you are almost there.

You can calculate the variance of $$\hat{\lambda}=\frac{n}{Y}$$ remembering that

$$Y=\Sigma_i X_i \sim \text{Inverse Gamma}$$

and thus you immediately solve the problem as its variance is a known parameter

...but you can also solve the problem using the integration, and calculating

$$\mathbb{V}\Bigg[\frac{1}{Y}\Bigg]=\mathbb{E}\Bigg[\frac{1}{Y^2}\Bigg]-\mathbb{E}^2\Bigg[\frac{1}{Y}\Bigg]$$

It is not difficult:

$$\mathbb{E}\Bigg[\frac{1}{Y}\Bigg]=\int_0^{\infty}\frac{1}{y}\frac{\lambda^n}{\Gamma(n)}y^{n-1}e^{-\lambda y}dy=\frac{\lambda}{n-1}\underbrace{\int_0^{\infty}\frac{\lambda^{n-1}}{\Gamma(n-1)}y^{(n-1)-1}e^{-\lambda y}dy}_{=1}=\frac{\lambda}{n-1}$$

and similarly for the second simple moment

Anyway this is not the asymptotic variance but it is the exact variance. To calculate the asymptotic variance you can use Delta Method

After simple calculations you will find that the asymptotic variance is $$\frac{\lambda^2}{n}$$ while the exact one is $$\lambda^2\frac{n^2}{(n-1)^2(n-2)}$$

• Perfect, I got the math, but now I have another question. How come does the asymptotic variance depend on n? According to what we have from your explanation, we would get: $\sqrt n (\hat{\lambda} - \lambda)\stackrel{D}{\rightarrow} \mathcal{N}(0, \sigma^{2})$ where: $\sigma^{2} = \lambda^{2} \dfrac{n^{2}}{(n-1)^{2} (n-2)}$ Commented Nov 24, 2020 at 2:24
• And also, I found alternatives explanations. It seems that we can also use the Cramer Rao lower bound. How does that fit here? Commented Nov 24, 2020 at 2:26
• @S.Cow . Yes, CR lower bound is the same of delta Method. Try, as an example, to find asymtotic and exact variance of $\hat{\frac{1}{\lambda}}$ Commented Nov 24, 2020 at 5:22
• So, when I do the asymptotic variance via the delta method, I get that the variance is $\lambda^{2}$. I'm a little confused, because it is not the same as the asymptotic variance $\dfrac{\lambda^{2}}{n}$ Commented Nov 24, 2020 at 15:28
• @S.Cow : that is not possible. Delta method and CR inequality have the same formula...just a different name . the denominator is the Fischer information... Commented Nov 24, 2020 at 15:40

Following the discussion with @tommik, I am still struggling with something:

Since $$\bar{X}$$ is the sample mean and given the properties of $$(X_{1},....X_{n})$$ in the problem set, the Central Limit Theorem tell us that:

$$\sqrt{n}\left(\bar{X} - \dfrac{1}{\lambda}\right) \stackrel{D}{\rightarrow} \mathcal{N}\left(0, \dfrac{1}{\lambda^{2}}\right)$$

We can apply the Delta Method:

Consider the function $$h(x) = x^{-1}$$, where $$h'(x)= -x^{-2}$$, therefore:

$$\sqrt{n}\left(h(\bar{X}) - h(\frac{1}{\lambda}) \right) \stackrel{D}{\rightarrow} \mathcal{N}\left(0, \overbrace{\left[h'(\frac{1}{\lambda})\right]^{2} \dfrac{1}{\lambda^{2}}}^{\sigma^{2}}\right)$$

$$\sqrt{n}\left(\hat{\lambda} - \lambda\right) \stackrel{D}{\rightarrow} \mathcal{N}\left(0, \left[h'(\frac{1}{\lambda})\right]^{2} \dfrac{1}{\lambda^{2}}\right)$$

Where notice that the argument inside of the function $$h'()$$ is $$\dfrac{1}{\lambda} = \lambda^{-1}$$.

Let's find out what the variance $$\sigma^{2}$$ is:

\begin{align*} \sigma^{2} &= \left[h'\left(\frac{1}{\lambda}\right)\right]^{2} \dfrac{1}{\lambda^{2}}\\ &= \left[ - \left(\dfrac{1}{\lambda}\right)^{-2} \right]^{2} \dfrac{1}{\lambda^{2}}\\ &= \dfrac{\lambda^{4}}{\lambda^{2}}\\ &= \lambda^{2}\\ \end{align*}

So, I don't know how to conciliate the comments and answers above.