For a Fisher Information, $\mathcal{I}(\theta)$, why does $\mathcal{I}(\theta) = n\mathcal{I}_1(\theta)$ not hold for multiple dimensions?

Suppose we have that $X_1, \ldots, X_n$ are iid from a distribution with ONE parameter, $\theta$. Then, under regulatory conditions, the Fisher Information may be written as:

$$\mathcal{I}(\theta) = - \operatorname{E} \left[\left. \frac{\partial^2}{\partial\theta^2} \log f(X;\theta)\right|\theta \right]$$

I know that for the iid case, we have that:

$$\mathcal{I}(\theta) = n\mathcal{I}_1(\theta)$$

Where $\mathcal{I}_1(\theta)$ is the Fisher Information for the first sample.

Now, I am wondering why this doesn't hold in a multi-variate setting, where instead of having ONE parameter, we have multiple.

Suppose now that $X_1, \ldots, X_n$ are iid from a distribution with parameters $\theta = (\theta_1, \ldots, \theta_k)$. In this case, it is my understanding that additivity will still hold, that is:

$$\mathcal{I}(\theta) = \mathcal{I}_1(\theta)+ \ldots + \mathcal{I}_n(\theta)$$

However, why do we not have that case that $\mathcal{I}(\theta) = n\mathcal{I}_1(\theta)$ anymore?

Thanks!

In the multi-parameter case, $\theta = (\theta_1, ..., \theta_k)$, Fisher information is a $k\times k$ matrix where the $ij$ entry is given by $$\mathcal{I}(\theta)_{ij} = -E[\frac{\partial}{\partial\theta_i \partial \theta_j} \log f(X;\theta)],$$ so, assuming that the sample is i.i.d, the additivity is still holds component-wise in the information matrix.
• Thanks, my question relates more to why in some cases we have $n$ times the first fisher information while in other cases we just add? – user136503 May 7 '16 at 21:53
• Then it should be an issue of dependence. If your sample is i.i.d, then $I_i(\theta) = I_j(\theta)$, for all $i, j$. Otherwise, if dependence is introduced, then $\sum_{i=1}^nI_i(\theta) \le nI(\theta)$. – V. Vancak May 7 '16 at 23:35