Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Suppose that $X$ is a random variable for which the p.d.f. or the p.f. is $f(x|\theta)$, where the value of the parameter $\theta$ is unknown but must lie in an open interval $\Omega$. Let $I_0(\theta)$ denote the Fisher information in $X.$ Suppose now that the parameter $\theta$ is replaced by a new parameter $\mu$, where $\theta = \psi(\mu)$ and $\psi$ is a differentiable function. Let $I_1(\mu)$ denote the Fisher information in $X$ when the parameter is regarded as $\mu.$ Show that $$I_1(\mu) = [\psi'(\mu)]^2 I_0[\psi(\mu)].$$

How would I do this? Do I need to use a Taylor expansion? Regardless, I would appreciate a written proof. This isn't for class but the above statement has been mentioned in texts without any detail whatsoever.

Thanks!

share|improve this question
1  
Hint: What's the definition of $I_0(\theta)$? Under standard regularity conditions, what is $\mathbb{E}\big(\frac{\partial \log f(x;\theta)}{\partial \theta}\big)$. Put this together with the chain rule of differentiation. –  cardinal May 8 '11 at 20:15

1 Answer 1

up vote 1 down vote accepted

By definition $I_{0}(\theta)=-\mathbb{E}\left[\frac{d^{2}\log f\left(X\vert\theta\right)}{d\theta^{2}}\right].$ So $I_{1}(\mu)=-\mathbb{E}\left[\frac{d^{2}\log f\left(X\vert\mu\right)}{d\mu^{2}}\right]$.By the chain rule we have $$I_{1}\left(\mu\right) = -\mathbb{E}\left[\frac{d^{2}\log f\left(X\vert\theta\right)}{d\theta^{2}}\left(\frac{d\theta}{d\mu}\right)^{2}+\frac{d\log f\left(X\vert\theta\right)}{d\theta}\frac{d^{2}\theta}{d\mu^{2}}\right]$$ $$= -\mathbb{E}\left[\frac{d^{2}\log f\left(X\vert\theta\right)}{d\theta^{2}}\right]\left(\frac{d\theta}{d\mu}\right)^{2}+\mathbb{E}\left[\frac{d\log f\left(X\vert\theta\right)}{d\theta}\right]\frac{d^{2}\theta}{d\mu^{2}}. $$

But $\mathbb{E}\left[\frac{d\log f\left(X\vert\theta\right)}{d\theta}\right]=0.$ So we get
$$ I_{1}\left(\mu\right)=I_{0}\left(\theta\right)\left(\frac{d\theta}{d\mu}\right)^{2}.$$

share|improve this answer
    
Can you explain how you're using the chain rule? Which functions are involved? –  Clark Kent May 9 '11 at 1:37
    
The functions involved are $\frac{d}{d\theta}\log f\left(X\vert\theta\right)$ and $\frac{d\theta}{d\mu}$ It is a product and you differentiating wrt $\mu$ –  Nana May 9 '11 at 3:10
    
NB: $\frac{d^{2}\log f\left(X\vert\mu\right)}{d\mu^{2}}=\frac{d}{d\mu}\left[\frac{d}{d\mu}\log f\left(X\vert\mu\right)\right]=\frac{d}{d\mu}\left[\frac{d}{d\theta}\log f\left(X\vert\theta\right)\cdot\frac{d\theta}{d\mu}\right]$ –  Nana May 9 '11 at 3:24
    
Where does 𝔼[d logf(X|θ) / dθ] = 0 come from? –  Daniel Mahler Sep 10 '13 at 20:43

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.