4
$\begingroup$

Let $y_1, \dots,y_n$ be i.i.d. random variables from $Exp(\theta)$, where $\theta$ is scale parameter. I have to find Fisher information $i(\theta)$.

The density function is $$f(y)=\frac{1}{\theta}e^{-\frac{y}{\theta}}$$

and the likelihood function $$L(\theta)=\frac{1}{\theta^n}e^{-\frac{\sum^{n}_{i=1}y_i}{\theta}}$$

The log-likelihood is

$$l(\theta)=-n\ln\theta-\frac{\sum^{n}_{i=1}y_i}{\theta}$$

Now, the score function $$l_*(\theta)=\frac{dl(\theta)}{d\theta}=-\frac{n}{\theta}+\frac{1}{\theta^2}\sum^{n}_{i=1}y_i$$

given the MLE

$$\hat \theta=\frac{\sum^{n}_{i=1}y_i}{n}$$

I differentiate again to find the observed information

$$j(\theta)=-\frac{dl_*(\theta)}{d\theta}=-(\frac{n}{\theta^2}-\frac{2}{\theta^3}\sum^{n}_{i=1}y_i)$$

and Finally fhe Fisher information is the expected value of the observed information, so

$$i(\theta)=\mathbb{E}(j(\theta))=-\frac{n}{\theta^2}+\frac{2}{\theta^3}n\theta=\frac{n}{\theta^2}$$

Is everything correct?

$\endgroup$
5
$\begingroup$

Yes it's correct. Very well done.

This doesn't simplify the work a lot in this case, but here's an interesting result . . . In the case of $n$ i.i.d. random variables $y_1,\dots,y_n$ , you can obtain the Fisher information $i_{\vec y}(\theta)$ for $\vec y$ via $n \cdot i_y (\theta$) where $y$ is a single observation from your distribution.

Here $\ell(\theta) = \ln( \frac{1}{\theta} e^{-y/\theta}) = -y/\theta - \ln(\theta) \implies \frac{\partial}{\partial \theta} \ell (\theta) = \frac{y}{\theta^2} - \frac{1}{\theta} \implies \frac{\partial^2}{\partial \theta^2} \ell(\theta) = - \frac{2y}{\theta^3} + \frac{1}{\theta^2}$ \begin{align*} i_y(\theta) &= - E \left[ \frac{\partial^2}{\partial \theta^2} \ell(\theta) \right] = -E \left[ - \frac{2y}{\theta^3} + \frac{1}{\theta^2} \right] = \dfrac{2 \theta}{\theta^3} - \dfrac{1}{\theta^2} = \dfrac{1}{\theta^2} \end{align*} and multiplying by $n$ gives Fisher information $n/\theta^2$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.