2
$\begingroup$

Let's consider a printer queue. We know that the expected number of printer jobs almost obeys a Poisson distribution, so $P_{\vartheta}(X=k)=e^{-\vartheta}\frac{\vartheta^k}{k!}$, where $\vartheta\in]0,\infty[$. We estimate the expected number of printer jobs $\vartheta$ by $\frac{1}{n}\sum\limits_{i=1}^nX_i$. Compute the Fisher information $I(\vartheta):=\mathbb{E}_{\vartheta}\left(\left(\frac{d\ln(P_{\vartheta}(X))}{d\vartheta}\right)^2\right)$.

We know that if $(X_1,\dots,X_n)$ are independent random variables with a distribution like $P_X(\vartheta)=f(X_1,\vartheta)\dots f(X_n,\vartheta)$, then

$$\mathbb{E}_{\vartheta}\left(\left(\frac{d\ln(P_{\vartheta}(X))}{d\vartheta}\right)^2\right)=n\cdot \mathbb{E}_{\vartheta}\left(\left(\frac{d\ln(P_{\vartheta}(X_i))}{d\vartheta}\right)^2\right).$$

If we consider the $n$-many independent observations $X:=(X_1,\dots,X_n)$ , where each $X_i$ is the number of printer jobs in a certain period of time, the probability is given by \begin{align*} &P_{\vartheta}(\{X=(x_1,\dots,x_n)\})=P_{\vartheta}(\{X_1=x_1\})\cdots P_{\vartheta}(\{X_n=x_n\})\\ &=\frac{e^{-\vartheta}\vartheta^{x_1}}{(x_1!)}\dots \frac{e^{-\vartheta}\vartheta^{x_n}}{(x_n!)}=\frac{e^{-n\vartheta}\vartheta^{\sum\limits_{i=1}^nx_i}}{\prod\limits_{i=1}^n(x_i!)}. \end{align*}

Applying the above statement yields after some manipulations $I(\vartheta)=\frac{n}{\vartheta}$.

However, the sample solution says:

\begin{align*} &I(\vartheta)=\sum\limits_{x=0}^{\infty}\frac{\left(\frac{d\ln(P_{\vartheta}(X))}{d\vartheta}\right)^2}{P_{\vartheta}(X)}=\sum\limits_{x=0}^{\infty}\frac{1}{x!}e^{\vartheta}\vartheta^{-x}\left(-e^{-\vartheta}\vartheta^x+e^{-\vartheta}x\vartheta^{x-1}\right)\\ &= \sum\limits_{x=0}^{\infty}\frac{\vartheta^x}{x!}e^{-\vartheta}\left(\frac{x}{\vartheta}-1\right)^2=\mathbb{V}(X)\frac{1}{\vartheta}^2=\frac{1}{\vartheta}. \end{align*}

This makes no sense to me? Why $x\to\infty$ and why is there only one random variable instead of a vector which represents $n$-many obersavtions?

$\endgroup$
3
  • $\begingroup$ You computed $I_n(\theta) = nI(\theta)$. $I(\theta)$ is what people mean by fisher information. $\endgroup$
    – Andrew
    Jan 17, 2023 at 17:46
  • $\begingroup$ @AndrewZhang, I don't exactly understand what you mean, maybe you can elaborate a bit. From what I understand the fisher information is defined with regard to a family of probability measures that refer to a random variable $X$. Howevr, in this particular case the random variable $X$ happens to be a vector of some random variables. What I computed is still $I(\theta)$ with reagrd to $X$. $\endgroup$
    – Philipp
    Jan 17, 2023 at 18:51
  • $\begingroup$ $I_n(\theta)$ is the variance of the score function, where the likelihood is computed from $n$ samples. However, people generally write this as $nI(\theta)$, since this makes clear the variance reduction from more samples. $\endgroup$
    – Andrew
    Jan 18, 2023 at 1:10

1 Answer 1

1
$\begingroup$

The Fisher information for all $N$ samples simplifies to $N$ times the Fisher information of a single sample. The score function is $l(\vartheta|x) = -\vartheta + x\log \vartheta, l'(\vartheta|x) = \frac{x}{\vartheta}-1, $

To compute Fisher information (of a single sample), we must compute the expectation over $X$ (hence the infinite sum) $$ I(\vartheta) =\mathbb{E} \left[ l'(\vartheta|x)^2 \right] =\sum_{k\ge 0} P_{\vartheta}(X=k) \left( \frac{k}{\vartheta}-1 \right)^2 =\mathbb{V} \left( \frac{X}{\vartheta} \right) =\frac{1}{\vartheta^2} \mathbb{V} \left( X \right) =\frac{1}{\vartheta} $$

$\endgroup$
1
  • $\begingroup$ Ah ok at least this answer why $x\to\infty$. Regarding the other issue I conclude from your answer that the sample solution is wrong as it only provides the Fisher information of a single sample and not of all samples. $\endgroup$
    – Philipp
    Jan 18, 2023 at 16:15

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .