Finding the Fisher information given the density

Question

For $X_1,...,X_n$ i.i.d. continuously distributed random variables with Lebesgue density $f_{\theta}(x)=\theta(\theta+1)x^{\theta-1}(1-x)\mathbf{1}_{(0,1)}(x)$ (Note: $\mathbf{1}_{(0,1)}(x)$ is an indicator function), How can I find the fisher information for a single observation?

For the Fisher information, I know the formula is $I(\theta)=-E_{\theta}\left[\frac{\partial^2}{\partial\theta^2}\ln f_{\theta}(X_1)\right]$ or alternatively $I(\theta)=E_{\theta}\left[(\frac{\partial}{\partial\theta}\ln f_{\theta}(X_1))^2\right]$. However, I can't solve for the partial derivative of $ln f_{\theta}(X_1)$ and how can I then find the expectation of this? I am very confused on this and I can't seem to find much information or examples of finding fisher information.

Answer 1

This is an exponential family. Therefore the second derivative of the score function is a function of $\theta$ only as you correctly observed. Therefore no painful integration! Actually this phenomena is characteristic of the models which are exponential families.

The Fisher information is $\frac{1}{\theta^2}+\frac{1}{(\theta+1)^2}$ for one observation.

Answer 2

The Fisher information for a random variable $X$ parameterized in terms of a single parameter $\theta$ is given by (assuming appropriate regularity conditions) $$ I_X(\theta)=-\mathsf E(\partial_\theta^2 \log f_X(X|\theta)). $$ Here, the expectation is taken with respect to the random variable $X$. For the specific case $$ f_X(x|\theta)=\theta(\theta+1)x^{\theta-1}(1-x)\mathbf{1}_{(0,1)}(x) $$ we compute the Fisher information for a single observation directly from the formula above. Since $$ \log f_X(X|\theta)=\log\theta+\log(\theta+1)+(\theta-1)\log X+\log(1-X) $$ we find $$ -\partial_\theta^2\log f_X(X|\theta)=\frac{1}{\theta^2}+\frac{1}{(\theta+1)^2}. $$ Notice that this expression does not contain $X$; thus, it is a constant and by the linearity of expectation $$ I_X(\theta)=\mathsf E(-\partial_\theta^2 \log f_X(X|\theta))=\mathsf E(\theta^{-2}+(\theta+1)^{-2})=\theta^{-2}+(\theta+1)^{-2}. $$ Furhtermore, Fisher information is additive so that if we have $n$ i.i.d. observations of $X$, i.e. $\mathbf X=\{X_1,\dots,X_n\}$, then the information contained in all $n$ observations is simply $$ I_{\mathbf X}(\theta)=nI_X(\theta)=\frac{n}{\theta^2}+\frac{n}{(\theta+1)^2}. $$