3
$\begingroup$

Can $$I(\theta) = -E\left[\frac{\partial^2}{\partial\theta^2} \ln f(x,\theta)\right] = 0$$ I've heard that Fisher's Information is strictly greater than zero but I'm unsure why. A proof would be much appreciated!

$\endgroup$
2
$\begingroup$

Show that the definition you've given is the same as $$ I(\theta) = \mathbb E\left[\left(\frac{\partial}{\partial\theta} \ln f(x\mid\theta)\right)^2\right] $$ without the "$\ln$".

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ This is wrong. The correct definition is with $\log$, see this wiki page and the proof therein. $\endgroup$ – Zhanxiong Nov 26 '15 at 6:39
2
$\begingroup$

The Fisher information is the variance of the score, given as $$ I(\theta) = \mathbb E\left[\left(\frac{\partial}{\partial\theta} \ln f(x\mid\theta)\right)^2\right], $$ which is nonnegative. For twice differentiable likelihoods, integration by parts yields the alternative formula given above, i.e., minus the expectation of the Hessian. For likelihoods that do not have two derivatives the alternative formula is not valid and can yield negative values. En example is the triangular distribution (continuous piecewise linear density $f$ on $[0,1]$, with $f(0)=f(1)=0$ and knot at $\theta\in(0,1)$) (see Berger, Bernardo, Sun, 2009).

| cite | improve this answer | |
$\endgroup$
0
$\begingroup$

From the Cramer-Rao bound, we know that variance of some unbiased estimator is lower bounded by reciprocal the fisher information. That is, $$Var(\hat{\theta})\geq \frac{1}{I(\theta)}.$$

Then we can get $I(\theta)\geq \frac{1}{Var(\hat{\theta})}$. Since the variance of some estimator should be $0<Var(\hat{\theta})<\infty$. We clearly have $0< I(\theta)< \infty$. Note that if $Var(\hat{\theta})\rightarrow\infty$, then we have $I(\theta)\rightarrow 0$.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ By going from $\operatorname{Var}(\hat{\theta})\geq \frac{1}{I(\theta)}$ to $I(\theta) \geq \frac{1}{\operatorname{Var}(\hat{\theta})}$, you're already assuming that $I(\theta)>0$, so I don't think that this proof is valid. $\endgroup$ – Dylan Nov 26 '15 at 6:58
  • $\begingroup$ Well. I think you should review the definition of $I(\theta)$. That is, $I(\theta) = E[(\frac{\partial \log f(x|\theta)}{\partial \theta})^2|\theta]$. This definition always has $I(\theta)\geq 0$. $\endgroup$ – aaronyxt Nov 26 '15 at 13:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.