6
$\begingroup$

In information geometry, the determinant of the Fisher information matrix is a natural volume form on a statistical manifold, so it has a nice geometrical interpretation.

But what is it in statistics? Does it measure anything meaningful? (For example, I would say that if it is zero, then the parameters are not independent. Does this go any further?)

Also, is there any closed form to compute it?

Thanks.

Update: I posted a similar question on stats.se.

$\endgroup$
1
  • 1
    $\begingroup$ The expected fisher information is called in this way because you can be interpret it as the expected curvature of the log-likelihood function. Greater values of the expected information imply a greater curvature of the log-likelihood, hence greater information about the true value of the parameter you are trying to estimate. $\endgroup$
    – Charlie
    Aug 29, 2014 at 15:31

2 Answers 2

6
$\begingroup$

If the log-likelihood is quadratic (i.e., the estimator is normally distributed), then the Fisher information is the reciprocal of the variance of the estimator (hence, the lower the variance of the estimator, the more "information" is provided by the data about the parameter...loosely speaking). The square root of the reciprocal of the Fisher Information is the standard error of an estimator (assuming approx. quadratic log-likelihood).

However, if the log-likelihood is not quadratic, then its reciprocal no longer represents the variance of the estimator. You either need to transform the estimator to give it a quadratic log-likelihood or use computational methods to determine the error.

$\endgroup$
3
$\begingroup$

You can generalize @Eupraxis1981's answer to all natural exponential families, that is distributions whose density has the form:

\begin{align} f(x \mid \eta) &= \exp\bigl({T(x) \cdot \eta - g(\eta)} + h(x)\bigr), \qquad x \in \Omega \end{align}

Their Fisher information is the Hessian of the (always strictly convex and $C^\infty$ differentiable) log-normalizer:

\begin{align} \mathcal I(\eta) &= -E(\nabla^2_{\eta} \log f(x \mid \eta) \mid \eta) \\ &= -E\Bigl(\nabla^2_{\eta} \bigl({T(x) \cdot \eta - g(\eta)} + h(x)\bigr) \mathrel{}\Bigm\vert\mathrel{} \eta\Bigr) \\ &= E\bigl(\nabla^2_{\eta} g(\eta) \mathrel{}\bigm\vert\mathrel{} \eta\bigr) \\ &= \nabla^2_{\eta} g(\eta). \end{align}

Therefore the curvature of the log-normalizer tells you how informative realizations of a distribution whose parameters are $\eta$ is. It makes sense that location parameters don't affect the Fisher information.

$\endgroup$
1
  • $\begingroup$ To learn more about this, search for Efron's papers on "statistical curvature". $\endgroup$ Aug 30, 2014 at 11:05

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .