# How to find the Fisher information for a function of a parameter, $I(g(\theta))$?

If $I(\theta)$ is the Fisher Information for $\theta$

Then how do I find $J=I(g(\theta ))$ ?

Currently my thoughts are that:

$$I(\theta) = E\left( \left(\frac{\partial}{\partial \theta}\log f(X;\theta)\right)^2 \right)$$ and if $u=g(\theta)$ where $g( \cdot )$ is differentiable, then $$\frac{\partial}{\partial u}=\frac{\partial\theta}{\partial u} \frac{\partial}{\partial \theta} =\frac{1}{g'(\theta)}\frac{\partial}{\partial \theta}$$ So $$I(u) = E\left( \left(\frac{\partial}{\partial u}\log f(X;u)\right)^2 \right) = E\left( \frac{1}{(g'(\theta))^2}\left(\frac{\partial}{\partial \theta}\log f(X;\theta)\right)^2 \right) \ \ \text{as u is a function of} \ \theta$$ but at this point I don't see the justification for taking the $\frac{1}{(g'(\theta))^2}$ out of the bracket...

I'm guessing the result should arrive at

$$I(g(\theta))=\frac{I(\theta)}{(g'(\theta))^2}$$

but I'm not sure how to get this...

• you can pull it out, the expectation is with respect to $X$ and $g'(\theta)$ is constant with respect to $X$ Commented Oct 4, 2017 at 4:18
• of course... Thank you!!! Commented Oct 4, 2017 at 4:19
• – glS
Commented Apr 16 at 21:34

The expectation operator is with respect to $X$ and $g'(\theta)$ is constant with respect to $X$ so it can be taken out of the operator -- giving the required result.
Technically, this formula should be stated using two different Fisher informations on the two sides: on one side we have the Fisher information associated to estimating $$\theta$$ from samples of $$X\sim p_\theta$$, and on the other side the Fisher information associated to estimating $$f(\theta)$$ from the same $$X$$ instead.
Say we want to estimate $$f(\theta)$$ from $$X\sim p_\theta$$, for some pdf $$x\mapsto p_\theta(x)$$. We can get the associated Fisher by thinking the pdf as a function of $$y=f(\theta)$$ (assuming $$f$$ is invertible and differentiable around $$\theta)$$, and thus compute $$I_y(y) = \mathbb{E}[(\partial_y \log p)^2] = \frac{\mathbb{E}[(\partial_\theta\log p)^2]}{f'(\theta)^2} = \frac{I_\theta(\theta)}{f'(\theta)^2}.$$ Note that the way to interpret $$I_y(y)$$ with $$y=f(\theta)$$ is that it tells you about the task of estimating the parameter $$\theta$$ from a distribution $$p_{f(\theta)}$$.
For example, suppose we want to estimate $$p$$ from $$X\sim\operatorname{Bern}(\sqrt p)$$, meaning $$\mathbb{P}(1|p)=\sqrt p$$. Then we get the Fisher $$I(p)= \frac{1}{4p} \frac{1}{p(1-p)},$$ where I used the fact that the Bernoulli distribution has Fisher $$I(p)=\frac{1}{p(1-p)}$$. Notice how this gets much smaller for $$p\approx0$$, and corresponds to using the (locally unbiased) estimator $$\hat p =-p + \binom{0}{2\sqrt p}.$$