14
$\begingroup$

Suppose that $X$ is a random variable for which the p.d.f. or the p.f. is $f(x|\theta)$, where the value of the parameter $\theta$ is unknown but must lie in an open interval $\Omega$. Let $I_0(\theta)$ denote the Fisher information in $X.$ Suppose now that the parameter $\theta$ is replaced by a new parameter $\mu$, where $\theta = \psi(\mu)$ and $\psi$ is a differentiable function. Let $I_1(\mu)$ denote the Fisher information in $X$ when the parameter is regarded as $\mu.$ Show that $$I_1(\mu) = [\psi'(\mu)]^2 I_0[\psi(\mu)].$$

How would I do this? Do I need to use a Taylor expansion? Regardless, I would appreciate a written proof. This isn't for class but the above statement has been mentioned in texts without any detail whatsoever.

Thanks!

$\endgroup$
1
  • 2
    $\begingroup$ Hint: What's the definition of $I_0(\theta)$? Under standard regularity conditions, what is $\mathbb{E}\big(\frac{\partial \log f(x;\theta)}{\partial \theta}\big)$. Put this together with the chain rule of differentiation. $\endgroup$
    – cardinal
    May 8, 2011 at 20:15

1 Answer 1

10
$\begingroup$

By definition $I_{0}(\theta)=-\mathbb{E}\left[\frac{d^{2}\log f\left(X\vert\theta\right)}{d\theta^{2}}\right].$ So $I_{1}(\mu)=-\mathbb{E}\left[\frac{d^{2}\log f\left(X\vert\mu\right)}{d\mu^{2}}\right]$.By the chain rule we have $$I_{1}\left(\mu\right) = -\mathbb{E}\left[\frac{d^{2}\log f\left(X\vert\theta\right)}{d\theta^{2}}\left(\frac{d\theta}{d\mu}\right)^{2}+\frac{d\log f\left(X\vert\theta\right)}{d\theta}\frac{d^{2}\theta}{d\mu^{2}}\right]$$ $$= -\mathbb{E}\left[\frac{d^{2}\log f\left(X\vert\theta\right)}{d\theta^{2}}\right]\left(\frac{d\theta}{d\mu}\right)^{2}+\mathbb{E}\left[\frac{d\log f\left(X\vert\theta\right)}{d\theta}\right]\frac{d^{2}\theta}{d\mu^{2}}. $$

But $\mathbb{E}\left[\frac{d\log f\left(X\vert\theta\right)}{d\theta}\right]=0.$ So we get
$$ I_{1}\left(\mu\right)=I_{0}\left(\theta\right)\left(\frac{d\theta}{d\mu}\right)^{2}.$$

$\endgroup$
5
  • 1
    $\begingroup$ Can you explain how you're using the chain rule? Which functions are involved? $\endgroup$
    – Glassjawed
    May 9, 2011 at 1:37
  • $\begingroup$ The functions involved are $\frac{d}{d\theta}\log f\left(X\vert\theta\right)$ and $\frac{d\theta}{d\mu}$ It is a product and you differentiating wrt $\mu$ $\endgroup$
    – Nana
    May 9, 2011 at 3:10
  • 1
    $\begingroup$ NB: $\frac{d^{2}\log f\left(X\vert\mu\right)}{d\mu^{2}}=\frac{d}{d\mu}\left[\frac{d}{d\mu}\log f\left(X\vert\mu\right)\right]=\frac{d}{d\mu}\left[\frac{d}{d\theta}\log f\left(X\vert\theta\right)\cdot\frac{d\theta}{d\mu}\right]$ $\endgroup$
    – Nana
    May 9, 2011 at 3:24
  • $\begingroup$ Where does 𝔼[d logf(X|θ) / dθ] = 0 come from? $\endgroup$ Sep 10, 2013 at 20:43
  • $\begingroup$ I think the chain rule in the comment to this answer by @Nana is incorrect. It should be $\frac{d}{d\mu}logf(X|\theta)$ in the left bracket. $\endgroup$
    – Thomas E
    Jul 12, 2017 at 16:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.