I have in literature saying the observed information $J(\theta)$ is equal to the Fisher information $I(\theta)$. They are given different donations and same parameter. It is not clear why if equal they have different donations. Could anyone please explain?


Let $X_1,...,X_n \sim f(x;\theta)$. Fisher information is a theoretical measure defined by $$ \mathcal{I}(\theta) = - \mathbb{E}\left[\frac{\partial^2}{\partial\theta^2}\ln f(x:\theta) \right], $$ where $\theta$ is the unknown parameter of interest, hence for sample of size $n$ and MLE $\hat{\theta}_n$, you can estimate the fisher information by $n\mathcal{I}(\hat{\theta}_n)$.

Observed information is defined by $$ \mathcal{I}_{obs}(\theta) = - n\left[\frac{1}{n}\sum_{i=1}^n\frac{\partial^2}{\partial^2 \theta}(\ln f(x_i:\hat{\theta}_n)) \right], $$
which is simply a sample equivalent of the above. So, as you can see, these two notions defined differently, however if you plug-in the MLE in fisher information you get exactly the observed information, $\mathcal{I}_{obs}(\theta)=n\mathcal{I}(\hat{\theta}_n)$.
To show it for a pretty general case, you can work out the algebra for a single parametric exponential family distribution (it is a straightforward calculations).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.