# Confusion about the definition of the Fisher information for discrete random variables

Suppose that $$X$$ has a absolutely continuous and positive probability density function $$f(x)$$, I know its Fisher information is defined through $$I(X) := \int_{\mathbb R} \frac{|f'|^2}{f} \mathrm{d}x.$$ However, I am very confused about the definition of $$I(X)$$ when $$X$$ is a non-negative integer-valued random variable with probability mass function $$(p_0,p_1,\ldots,p_n,\ldots)$$. In most references I have found so far, it seems that the definition of the Fisher information contains some unknown parameter $$\theta$$ (for example, if $$X$$ is Bernoulli distributed, then the parameter is the success probability in each trial). Since I am not familiar with statistics, I am very confused as to how should we define Fisher information $$I(X)$$ when $$X$$ is a non-negative integer-valued random variable with (unknown) probability mass function $$(p_0,p_1,\ldots,p_n,\ldots)$$. Basically, my motivation is that I have a Fokker-Planck equation for the evolution of the probability mass function of a non-negative integer-valued stochastic process, and I want to know how its Fisher information evolves in time. Thanks for any help or reference!

• Fisher information is a function of a parameter when a family of probability distributions is parametrized. Dec 23, 2020 at 19:18
• For any probability density $f$, I can clearly define its Fisher information via $I(f) = \int_{\mathbb R} \frac{|f'|^2}{f} \mathrm{d}x$ (set to $\infty$ if this integral doesn't make sense), in this case, there is no "parameter" at all Dec 23, 2020 at 20:38

For a discrete known probability mass function, there is no parameter $$\theta$$—you know the full distribution. If however you know just the type or form distribution (such as a Gaussian, Bernoulli, etc.), you need to know the parameters (such as the sufficient statistics) in order calculate the Fisher Information (and other measures). After all, the Fisher Information (and the mean, and the variance, and...) of a Gaussian distribution depends upon the mean and the standard deviation, which in your terminology is $$\theta$$.
$$I[{\bf P}] = \sum\limits_{i=1}^n \frac{(p_{i} - p_{i-1})^2}{p_i}$$
• Hi, thanks for your answer, but it is still vague to me. For a known probability density function $f(x)$ (let's say it's positive everywhere and absolutely continuous), I know its Fisher information is defined via $I(f) = \int_{\mathbb R} \frac{|f'|^2}{f} \mathrm{d}x$, there is no parameter as well. What would be the definition of Fisher information in the case of a general (unknown) discrete probability mass function $(p_0,p_1,\ldots,)$ then (with no parameter since it is a general pmf)? Dec 23, 2020 at 20:36
• Yep.$\mbox{}\mbox{}\mbox{}$ Dec 28, 2020 at 6:44