My provided definition of the Fisher information $\mathcal{I}(\theta)$ is the expected value of the observed information $I(\theta)$, where $I(\theta)$ is the second derivative of the log-likelihood function of some distribution $F(x|\theta)$.

There is an exercise in my notes which mentions that the median of the exponential distribution can be estimated using maximum likelihood, using

$$ Q(p=0.5|\hat{\theta}) $$

where $Q$ is the quartile function. The exercise asks for me to find the Fisher information for this quantity.

I can't really understand what this means. Clearly $$ Q(p=0.5|\hat{\theta}) = \theta^{-1} \ln{2} $$ which I understand to be the median, but now what is the Fisher information? I don't think I have a distribution, do I? $\theta$ is the true value of the parameter, so it seems to me like I have an unknown quantity here... which leaves me with no idea how to derive the Fisher information.

I'm not looking for a drawn out answer if that's what is required here, but rather just an idea of what is meant by the Fisher information in this context.




Notice that the quantile function is a 1-to-1 transformation of $\theta$; therefore, we can invoke the invariance property of maximum likelihood estimators (see p16 here). This property implies that the log-likelihood of a particular median estimate (given $\theta$) is equal to the log-likelihood of $\theta$ (i.e., $\mathcal{L}_{\theta}(\theta)$), which allows us to derive the log-likelihood for the median $\mathcal{L}_m(m)$ by substitution and transformation:

$$\mathcal{L}_m(m) = \mathcal{L}_{\theta}\left(\frac{\ln{2}}{m}\right) \implies \mathcal{I}(\hat{m}) = E\left[\frac{d^2}{dm^2}\mathcal{L}_{\theta}\left(\frac{\ln{2}}{m}\right)_{m=\hat{m}}\right]$$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.