The Fisher information matrix $I(\theta;X)$ about $\theta$ based on $X$ is defined as the matrix with elements $$I_{i,j}(\theta;X) = \operatorname{Cov}_\theta\bigg(\frac{\partial}{\partial \theta_i}\log f_X(X\mid\theta), \frac{\partial}{\partial\theta_i}\log f_X(X\mid \theta) \bigg).$$

Exercise: Let $X\sim \text{Pois}(\theta)$. Show that $I(\theta;X) = 1/\theta$.

What I've tried: If I'm not mistaken then $\dfrac{\partial}{\partial \theta_i} \log f_X(X \mid \theta) = \dfrac{x_i}{\theta} + \log e$. The Fisher information matrix (a $1\times 1$ matrix in this example) would be given by $\operatorname{Cov}_\theta\left(\dfrac{x_i} \theta + \log e, \dfrac{x_i} \theta + \log e\right)$. We know that $\operatorname{Cov}_\theta\left(\dfrac{x_i} \theta + \log e, \dfrac{x_i}{\theta} + \log e\right) = \operatorname{Var}_\theta\left(\dfrac{x_i}\theta + \log e\right) = \operatorname{Var}_\theta \left(\dfrac{x_i}{\theta}\right) = x_i^2 \operatorname{Var}_\theta \left(\dfrac 1 \theta \right).$ Obviously I'm doing something wrong as the $x_i^2$ before the variance is a problem. Besides that, I'm not sure if I can calculate $\operatorname{Var}_\theta\left(\dfrac 1 \theta \right)$.

Question: How do I show that $I(\theta;X) = 1/\theta$?

Thanks in advance!

  • $\begingroup$ $\operatorname{Var}_\theta \left(\dfrac{x_i}{\theta}\right) = x_i^2 \operatorname{Var}_\theta \left(\dfrac 1 \theta \right)$ doesn't make sense. You need $\operatorname{Var}_\theta \left(\dfrac{x_i}{\theta}\right) = \dfrac 1 {\theta^2} \operatorname{Var}_\theta (x_i). \qquad$ $\endgroup$ – Michael Hardy Dec 11 '17 at 16:52
  • $\begingroup$ @MichaelHardy Why is that so? It's the variance w.r.t. $\theta$ right? Don't we use $\text{Var}_X$ so show that the variance is to be taken w.r.t. $X$? $\endgroup$ – titusAdam Dec 11 '17 at 16:59
  • $\begingroup$ $\theta$ is a parameter and so it is just a constant. The notation $E_\theta$ or $\operatorname{Var}_\theta$ just lists the relevant parameter(s) in the subscript. The random quantities are still $X$ and all things derived from it. $\endgroup$ – Kim Jong Un Dec 11 '17 at 17:02
  • $\begingroup$ @KimJongUn What would relevant mean in this sense? I'd say it's rather confusing. When taking the expectation $\operatorname{E}_X$ we mean that we take the expectation wrt $X$ right? $\endgroup$ – titusAdam Dec 11 '17 at 17:05
  • $\begingroup$ @titusAdam : No. I don't take expected values or variances "with respect to" anything. Rather $\operatorname{Var}_\theta$ means the variance given that the value of the parameter is $\theta. \qquad$ $\endgroup$ – Michael Hardy Dec 11 '17 at 17:05

The mistake is in the last step: $$ \operatorname{Var}(x_i/\theta) {{}\color{red}={}} x_i^2\operatorname{Var}(1/\theta). $$ Instead, it should be $$ \operatorname{Var}(x_i/\theta)=\operatorname{Var}(x_i)/\theta^2=\theta/\theta^2=1/\theta. $$

  • $\begingroup$ When you used \text{Var} instead of \operatorname{Var} then you don't get proper spacing in things like $3\operatorname{Var}(X)$ or $3\operatorname{Var} X.$ Instead you see $3\text{Var}(X)$ or $3\text{Var} X.$ I edited accordingly. $\qquad$ $\endgroup$ – Michael Hardy Dec 11 '17 at 16:55
  • $\begingroup$ @MichaelHardy Thanks Mike. Glad to learn something new. $\endgroup$ – Kim Jong Un Dec 11 '17 at 16:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.