# Fisher information of exponential distribution using the generic formula.

I am trying to calculate Fisher information when $X_1...X_n$ are IID and follow the exponential distribution:

$$f(x)=\frac{1}{\theta}e^{-\frac{x}{\theta}}$$

I use the following formula of Fisher information to confirm that the result is indeed the same as with the other formulas:

$$I(\theta) = E[(\frac{\partial}{\partial \theta }logf(X;\theta))^2]$$

I have calculated that for $\frac{\partial}{\partial \theta }logf(X;\theta)$ we have:

$$\frac {-n}{\theta}+ \frac{\sum_1^nX_i}{\theta^2}$$

Squaring that:

$$\frac{n^2}{\theta^2}+\frac{(\sum_1^n X_i)^2}{\theta^4}-\frac{2n\sum_1^nX_i}{\theta^3}$$

And finally taking the expected value and letting it pass through the constants::

$$\frac{n^2}{\theta^2}+\frac{E(\sum_1^n X_i)^2}{\theta^4}-\frac{2nE(\sum_1^nX_i)}{\theta^3}$$

Recalling that $E(X)= \theta$ and $E(X^2)=2\theta^2$ it seems as though the 2nd and 3rd term cancel out leaving $\frac{n^2}{\theta^2}$, but the correct answer is $\frac{n}{\theta^2}$. Where does this go wrong?

Ok using that parameterisation I agree your likelihood is correct! So method one we differentiate again to get $$\ell_{\theta \theta} = -\frac{2 \sum x_i }{\theta^3} + \frac{n}{\theta^2}$$ now since $\mathbb{E} \sum_i x_i = n \theta$ we get \begin{align} \mathcal{I} &= - \mathbb{E}\left[ \ell_{\theta \theta} \right] \\ &= 2 \frac{n\theta}{\theta^3} - \frac{n}{\theta^2} \\\ &= \frac{n}{\theta^2} \end{align} Alternatively noting that \begin{align} \mathbb{E}(\sum_i X_i )^2 &= n\mbox{Var}(X_1) + (n\theta)^2 \\ &= n \theta^2 + n^2 \theta^2 \end{align} we have \begin{align} \mathbb{E} \left[ \ell_{\theta} ^2 \right] &= \mathbb{E} \left[ \left(\frac{1}{\theta^2} \sum_i x_i - \frac{n}{\theta} \right)^2\right] \\ &= \frac{1}{\theta^4}\mathbb{E} \left[\left(\sum_i X_i\right)^2 \right]- \frac{2n}{\theta^3}\mathbb{E}\left[\sum X_i \right]+ \frac{n^2}{\theta^2} \\ &= \frac{n\theta^2}{\theta^4} + \frac{n^2 \theta^2}{\theta^4} - \frac{2 n^2 \theta}{\theta^3} + \frac{n^2}{\theta^2} \\ &=\frac{n}{\theta^2}+\frac{2 n^2}{\theta^2} - \frac{2n^2}{\theta^2} \\ &= \frac{n}{\theta^2}. \end{align}
• I am using the alternate specification for the distribution, where $\theta = \frac{1}{\theta}$. But the same problem occurs for me on this specification as well if the Fisher formula provided in the question is used, instead of the 2nd derivative based one.
• Hope that helps, I wonder if maybe where you went wrong was in calculating $\mathbb{E} (\sum_i x_i )^2$? Mar 7, 2017 at 12:04