0
$\begingroup$

I am asked to find the fisher information contained in $X_1 \sim N(\theta_1, \theta_2)$ (ie: two unknown parameters, only one observation). How would I find the Fisher information here?

I know that with a sample $X_1,X_2,\ldots,X_n $~$N(\mu,\sigma^2)$ and $\sigma^2=1$, Fisher's information is given by : $$ -E(\frac{d^2}{d\mu^2} \ln f(x))=1/\sigma^2. $$ Though this is the case with one paramter and I am not sure how it would map on to the case with two parameters. I imagine there is some use of a Hessian but I am not sure what to do.

$\endgroup$
1
$\begingroup$

It will be the expected value of the Hessian matrix of $\ln f(x;\mu, \sigma^2)$. Specifically for the normal distribution, you can check that it will a diagonal matrix. The $\mathcal{I}_{11}$ you have already calculated. For the second diagonal term $$ \ln f(x;\mu, \sigma)=-\frac{1}{2}\ln(2 \sigma^2)+\frac{1}{2\sigma^2}(x-\mu)^2, $$ $$ l'_{\sigma^2} = - \frac{1}{2\sigma^2} - \frac{1}{2\sigma^4}(x-\mu)^2, $$ hence $$ \mathcal{I}_{22}= -\mathbb{E}[l''_{\sigma^2}] = - \mathbb{E} [ \frac{1}{2\sigma^4} - \frac{1}{\sigma^6}(x-\mu)^2] = -\frac{1}{2\sigma^4} + \frac{2}{\sigma^4} = \frac{1}{2\sigma^4} . $$ And for the non-diagonal terms $$ \mathcal{I}_{22}= -\mathbb{E}[l''_{\sigma^2,\mu}] = - \mathbb{E}\frac{2(x-\mu)}{2\sigma^4} = 0. $$

$\endgroup$
  • $\begingroup$ Can you say a bit more about that? What would be the entries in the Hessian? $\endgroup$ – user617621 Mar 10 at 1:12
  • $\begingroup$ If you let $l$ be the log-likelihood function and write $v\equiv \sigma^2$ for simplicity, the Hessian of the log-likelihood function will be equal to $$\color{blue}{\begin{bmatrix}\frac{\partial^2 l}{\partial \mu^2} & \frac{\partial^2 l}{\partial \mu \partial v} \\ \frac{\partial^2 l}{\partial \mu \partial v} & \frac{\partial^2 l}{\partial v^2}\end{bmatrix}}.$$Note that the Hessian is a symmetric matrix, so once you calculate the top right entry, you can copy it into the bottom left entry. $\endgroup$ – Minus One-Twelfth Mar 10 at 4:13
  • $\begingroup$ @jdoe pls see the edited answer $\endgroup$ – V. Vancak Mar 10 at 8:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy