I am trying to calculate Fisher information when $X_1...X_n$ are IID and follow the exponential distribution:


I use the following formula of Fisher information to confirm that the result is indeed the same as with the other formulas:

$$I(\theta) = E[(\frac{\partial}{\partial \theta }logf(X;\theta))^2]$$

I have calculated that for $\frac{\partial}{\partial \theta }logf(X;\theta)$ we have:

$$\frac {-n}{\theta}+ \frac{\sum_1^nX_i}{\theta^2}$$

Squaring that:

$$\frac{n^2}{\theta^2}+\frac{(\sum_1^n X_i)^2}{\theta^4}-\frac{2n\sum_1^nX_i}{\theta^3}$$

And finally taking the expected value and letting it pass through the constants::

$$\frac{n^2}{\theta^2}+\frac{E(\sum_1^n X_i)^2}{\theta^4}-\frac{2nE(\sum_1^nX_i)}{\theta^3}$$

Recalling that $E(X)= \theta$ and $E(X^2)=2\theta^2$ it seems as though the 2nd and 3rd term cancel out leaving $\frac{n^2}{\theta^2}$, but the correct answer is $\frac{n}{\theta^2}$. Where does this go wrong?


1 Answer 1


Ok using that parameterisation I agree your likelihood is correct! So method one we differentiate again to get $$ \ell_{\theta \theta} = -\frac{2 \sum x_i }{\theta^3} + \frac{n}{\theta^2} $$ now since $\mathbb{E} \sum_i x_i = n \theta$ we get $$ \begin{align} \mathcal{I} &= - \mathbb{E}\left[ \ell_{\theta \theta} \right] \\ &= 2 \frac{n\theta}{\theta^3} - \frac{n}{\theta^2} \\\ &= \frac{n}{\theta^2} \end{align} $$ Alternatively noting that $$ \begin{align} \mathbb{E}(\sum_i X_i )^2 &= n\mbox{Var}(X_1) + (n\theta)^2 \\ &= n \theta^2 + n^2 \theta^2 \end{align} $$ we have $$ \begin{align} \mathbb{E} \left[ \ell_{\theta} ^2 \right] &= \mathbb{E} \left[ \left(\frac{1}{\theta^2} \sum_i x_i - \frac{n}{\theta} \right)^2\right] \\ &= \frac{1}{\theta^4}\mathbb{E} \left[\left(\sum_i X_i\right)^2 \right]- \frac{2n}{\theta^3}\mathbb{E}\left[\sum X_i \right]+ \frac{n^2}{\theta^2} \\ &= \frac{n\theta^2}{\theta^4} + \frac{n^2 \theta^2}{\theta^4} - \frac{2 n^2 \theta}{\theta^3} + \frac{n^2}{\theta^2} \\ &=\frac{n}{\theta^2}+\frac{2 n^2}{\theta^2} - \frac{2n^2}{\theta^2} \\ &= \frac{n}{\theta^2}. \end{align} $$

  • $\begingroup$ I am using the alternate specification for the distribution, where $\theta = \frac{1}{\theta}$. But the same problem occurs for me on this specification as well if the Fisher formula provided in the question is used, instead of the 2nd derivative based one. $\endgroup$
    – Dole
    Mar 7, 2017 at 2:32
  • $\begingroup$ Hope that helps, I wonder if maybe where you went wrong was in calculating $\mathbb{E} (\sum_i x_i )^2 $? $\endgroup$
    – Nadiels
    Mar 7, 2017 at 12:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.