I am viewing an example of finding the Fisher information for a single sampling from an exponential distribution where: $$P(x|\theta) = \frac{1}{\theta}e^{-\frac{x}{\theta}}$$ The score $S$ is $S(x|\theta) = \frac{\partial}{\partial\theta}logP(x|\theta) = -\frac{1}{\theta} + \frac{x}{\theta^2}$. Fisher information is the expectency of $S^2$ which is: $$E_x[S^2] = E_x\left[\frac{1}{\theta^2} - 2\frac{x}{\theta^3} + \frac{x^2}{\theta^4}\right]$$

I know this might sound strange, but I don't know how to calculate this expectation. Something is mixed for me here. I know that $$E[P(x)]=\int xp(x)dx$$

But I can't connect the two pieces of information.

In the book, they got: $$E_x[S^2] = E_x\left[\frac{1}{\theta^2} - 2\frac{x}{\theta^3} + \frac{x}{\theta^4}] = \frac{1}{\theta^2} - 2\frac{\theta}{\theta^3} + \frac{2\theta^2}{\theta^4}\right] = \frac{1}{\theta^2}$$

But I can't see how they got that.

Any information will be useful. Thanks.


Since $X$ is an exponential distribution with parameter $\frac{1}{\theta}$, that is, $X\sim\text{exp}(\frac{1}{\theta})$, its mean is given by

$$E[X] = \frac{1}{\frac{1}{\theta}} = \theta,$$

and the variance is given by

$$\text{var}(X) = \frac{1}{\left(\frac{1}{\theta}\right)^2} = \theta^2.$$

By other hand, we have that

$$E[X^2] = \text{var}(X) + (E[X])^2 = 2\theta^2$$.

In addition, you can use the linearity property of the expected value

\begin{align} E_x[S^2] &= E_x\left[\frac{1}{\theta^2} - 2\frac{X}{\theta^3}+ \frac{X^2}{\theta^4}\right]\\ &= \frac{1}{\theta^2} - \frac{2}{\theta^3}E[X]+ \frac{1}{\theta^4}E[X^2] \end{align}

where we have also used the fact that $\theta$ is a constant (a deterministic parameter) and therefore $E\left[\frac{1}{\theta^2}\right] = \frac{1}{\theta^2}$.

  • $\begingroup$ This is so more clear now... Thank you! $\endgroup$ – MathBgu Nov 17 '15 at 13:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.