1
$\begingroup$

Up until a few days ago I was thinking that the following two forms of the Fisher Information are "always" equivalent: $$(1) \quad \mathcal{I(\theta)}= E_\theta [\frac{\partial \log \ell(y;\theta)}{\partial \theta} \frac{\partial \log \ell(y; \theta)}{\partial \theta'}],$$ $$(2) \quad \mathcal{I(\theta)}= E_\theta [-\frac{\partial^2 \log \ell(y;\theta)}{\partial \theta\partial \theta'} ],$$ where $\theta$ is the parameter vector, $y$ is the vector of observations, and $\ell$ is the likelihood function.

I heard that (1) is the original definition of the Fisher Information, however (2) only holds when the model is not "misspecified".

I consulted some sources, among them Statistics and Econometrics Models (Gourieroux and Monfort 1995, Vol 1, p83 property 3.8), but there is no mention of any regularity conditions needed to hold for the second definition of $\mathcal{I}(\theta)$ to be equivalent to the original definition (1).

How does misspecification of a model cause this equivalency to break?

$\endgroup$
1
  • $\begingroup$ What they actually do to get the second equation is differentiating under the integral sign which is not always allowed. Read the conditions here: en.m.wikipedia.org/wiki/Leibniz_integral_rule $\endgroup$
    – Shashi
    Commented Nov 22, 2017 at 21:36

1 Answer 1

1
$\begingroup$

There are regularity conditions so that the two expressions are equal. In Statistical Theory of F.Abramovich and Y. Ritov, they give these two regularity conditions:

  1. The support of $f(x,\theta)$, namely $K:=\{x\ | \ f(x,\theta)>0 \ \}$, does not depend on $\theta$.
  2. One has $$\frac{d}{d\theta}\int_{K} g(x) f(x,\theta)dx=\int_{K} g(x)\frac{\partial}{\partial\theta}f(x,\theta)dx$$ for any $g(\cdot)$ for which $E[|g(X)|]$ is finite.

So actually one must fulfill the conditions of differentiating under the integral sign. As you might know one gets the other form of the Fisher Information which is not always right by differentiating under the integral sign, see for example this page.

I'll show you an example where both conditions are not fulfilled and how it goes wrong.

Example.

Let $X\sim \text{Unif}[0,\theta]$. So $f(x,\theta) = \frac{1}{\theta}\textbf{1}_{[0,\theta]}$. The support is $[0,\theta]$ and therefore dependent on $\theta$. We got a problem with the first condition. Now let's look to the second condition. Take $g(x)=1$: \begin{align} \frac{d}{d\theta}\int_0^\theta f(x,\theta)dx=\frac{d}{d\theta}\int_0^\theta \frac{1}{\theta}dx = \frac{d}{d\theta} 1 =0 \end{align}

On the other hand: \begin{align} \int_0^\theta \frac{\partial}{\partial\theta} f(x,\theta)dx = \int_0^\theta -\frac{1}{\theta^2} dx \end{align} So: $$\frac{d}{d\theta}\int_0^\theta f(x,\theta)dx\neq\int_0^\theta \frac{\partial}{\partial\theta}f(x,\theta)dx$$ So condition 2 is also not fullfilled. Now calculate the fisher number: \begin{align} E\bigg[\left(\frac{\partial}{\partial\theta}\log f(X,\theta) \right) ^2 \bigg] =E\bigg[\left(\frac{\partial}{\partial\theta}-\log\theta \right) ^2 \bigg] = \frac{1}{\theta^2} \end{align} On the other hand: \begin{align} E\bigg[-\frac{\partial^2}{\partial\theta^2}\log f(X,\theta) \bigg] = E\bigg[\frac{\partial^2}{\partial\theta^2}\log\theta \bigg] = -\frac{1}{\theta^2} \end{align} And so the expressions are not equal.

$\endgroup$
2
  • 1
    $\begingroup$ thank you your example was really illuminating. $\endgroup$ Commented Nov 24, 2017 at 20:06
  • $\begingroup$ @Xiang glad to help you! $\endgroup$
    – Shashi
    Commented Nov 24, 2017 at 20:09

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .