0
$\begingroup$

Up until a few days ago I was thinking that the following two forms of the Fisher Information are "always" equivalent: $$(1) \quad \mathcal{I(\theta)}= E_\theta [\frac{\partial \log \ell(y;\theta)}{\partial \theta} \frac{\partial \log \ell(y; \theta)}{\partial \theta'}],$$ $$(2) \quad \mathcal{I(\theta)}= E_\theta [-\frac{\partial^2 \log \ell(y;\theta)}{\partial \theta\partial \theta'} ],$$ where $\theta$ is the parameter vector, $y$ is the vector of observations, and $\ell$ is the likelihood function.

I heard that (1) is the original definition of the Fisher Information, however (2) only holds when the model is not "misspecified".

I consulted some sources, among them Statistics and Econometrics Models (Gourieroux and Monfort 1995, Vol 1, p83 property 3.8), but there is no mention of any regularity conditions needed to hold for the second definition of $\mathcal{I}(\theta)$ to be equivalent to the original definition (1).

How does misspecification of a model cause this equivalency to break?

$\endgroup$
0
$\begingroup$

There are regularity conditions so that the two expressions are equal. In Statistical Theory of F.Abramovich and Y. Ritov, they give these two regularity conditions:

  1. The support of $f(x,\theta)$, namely $K:=\{x\ | \ f(x,\theta)>0 \ \}$, does not depend on $\theta$.
  2. One has $$\frac{d}{d\theta}\int_{K} g(x) f(x,\theta)dx=\int_{K} g(x)\frac{\partial}{\partial\theta}f(x,\theta)dx$$ for any $g(\cdot)$ for which $E[|g(X)|]$ is finite.

So actually one must fulfill the conditions of differentiating under the integral sign. As you might know one gets the other form of the Fisher Information which is not always right by differentiating under the integral sign, see for example this page.

I'll show you an example where both conditions are not fulfilled and how it goes wrong.

Example.

Let $X\sim \text{Unif}[0,\theta]$. So $f(x,\theta) = \frac{1}{\theta}\textbf{1}_{[0,\theta]}$. The support is $[0,\theta]$ and therefore dependent on $\theta$. We got a problem with the first condition. Now let's look to the second condition. Take $g(x)=1$: \begin{align} \frac{d}{d\theta}\int_0^\theta f(x,\theta)dx=\frac{d}{d\theta}\int_0^\theta \frac{1}{\theta}dx = \frac{d}{d\theta} 1 =0 \end{align}

On the other hand: \begin{align} \int_0^\theta \frac{\partial}{\partial\theta} f(x,\theta)dx = \int_0^\theta -\frac{1}{\theta^2} dx \end{align} So: $$\frac{d}{d\theta}\int_0^\theta f(x,\theta)dx\neq\int_0^\theta \frac{\partial}{\partial\theta}f(x,\theta)dx$$ So condition 2 is also not fullfilled. Now calculate the fisher number: \begin{align} E\bigg[\left(\frac{\partial}{\partial\theta}\log f(X,\theta) \right) ^2 \bigg] =E\bigg[\left(\frac{\partial}{\partial\theta}-\log\theta \right) ^2 \bigg] = \frac{1}{\theta^2} \end{align} On the other hand: \begin{align} E\bigg[-\frac{\partial^2}{\partial\theta^2}\log f(X,\theta) \bigg] = E\bigg[\frac{\partial^2}{\partial\theta^2}\log\theta \bigg] = -\frac{1}{\theta^2} \end{align} And so the expressions are not equal.

$\endgroup$
  • 1
    $\begingroup$ thank you your example was really illuminating. $\endgroup$ – math_enthusiast Nov 24 '17 at 20:06
  • $\begingroup$ @Xiang glad to help you! $\endgroup$ – Shashi Nov 24 '17 at 20:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.