# Comparing Fisher Information of sample to that of statistic

Let $X_1,...,X_n$ be Bernoulli($p$) where $p$ is unknown, and $n>2$, and let $T=X_1+X_2$. My task is to calculate the information about $p$ in the entire sample and compare it to the information about $p$ given by the statistic.

After a few lines of work, I obtain the following expression for information contained in the sample: $$I_X(p)= n*E_p[(\frac{X_1p^{-1}(1-p)+X_1+1}{1-p})^2]$$

To calculate this, I first calculated the information given by one observation, and then multiplied that information by $n$.

Now, after some work, I obtained the following expression for information contained in $T$: $$I_T(p)=E_p[(\frac{Tp^{-1}(1-p)-1}{1-p})^2]$$

Now, I know from previous questions that $T$ is sufficient for $p$. Hence, $I_X(p)$ should equal $I_T(p)$. However, I have no idea how I am to compare these two quantities, because one has an $n$, and the other has an $X_2$ (in the $T$). Any advice?

• Are you sure of the $I_X(p)$ expression? I have a hard time believing it's correct. – Ahmad Bazzi Sep 5 '18 at 19:09

Fisher Information Matrix (FIM), is the negative of the Expectation of the Hessian of the log likelihood function, namely $$I(p) = - E H(p)$$ where $I(p)$ is the FIM and $H(p)$ is the Hessian. Your likelihood function for independent samples is $$L(p) = f(x_1\vert p) \ldots f(x_n \vert p)$$ where $$f(x_i \vert p) = p^{x_i} (1 - p)^{1 - x_i}$$ Both equations above give us the likelihood function $$L(p) = p^{ \sum x_i} (1 - p)^{n - \sum x_i}$$ Log likelihood becomes $$l(p) = \log L(p) = \log \big( p^{ \sum x_i} (1 - p)^{n - \sum x_i} \big)$$ Denoting $\bar{x} = \frac{1}{n} \sum x_i$ for sake of simple presentation, one gets: $$l(p) = n\bar{x} \log(p) + n(1 - \bar{x})\log(1-p)$$ The score function (gradient of the log likelihood) is $$s(p) = l'(p) = n \frac{\bar{x}}{p} - n \frac{(1- \bar{x})}{1 - p}$$ The Hessian becomes $$H(p) = s'(p) = - \frac{n(1-2p)\bar{x} + np^2}{p^2(1-p)^2}$$ The FIM becomes $$I(p)= -E H(p) = E\big[ \frac{n(1-2p)\bar{x} + np^2}{p^2(1-p)^2} \big] = \big[ \frac{n(1-2p)E\bar{x} + np^2}{p^2(1-p)^2} \big]$$ But $E\bar{x} = p$ so $$I(p) = E\big[ \frac{n(1-2p)\bar{x} + np^2}{p^2(1-p)^2} \big] = \big[ \frac{n(1-2p)p + np^2}{p^2(1-p)^2} \big] = \frac{n}{p(1-p)}$$ Now, $$T = X_1 + X_2$$ and the distribution of $T$ is $$g(T=t\vert p) = Pr(T = t) = C_2^t p^t(1-p)^{n-t}$$ Using Bayes theorem, we know that $$f(x_1 \ldots x_n \vert T = t, p) = \frac{f(x_1 \ldots x_n,t=T \vert p)}{g(T= t \vert p)}$$ Let's get $f(x_1 \ldots x_n,t=T \vert p)$, the likelihood function becomes $$f(x_1 \ldots x_n , T = t\vert p) = p^{t + \sum\limits_{i=3}^n x_i}(1-p)^{n - t - \sum\limits_{i=3}^n x_i}$$ $$f(x_1 \ldots x_n \vert T = t, p) = \frac{f(x_1 \ldots x_n,t=T \vert p)}{g(T= t \vert p)}$$ = \frac{p^{t + \sum\limits_{i=3}^n x_i}(1-p)^{n - t - \sum\limits_{i=3}^n x_i}}{C_2^t p^t(1-p)^{n-t}} which is $$f(x_1 \ldots x_n \vert T = t, p) = \frac{p^{ \sum\limits_{i=3}^n x_i}(1-p)^{ - \sum\limits_{i=3}^n x_i}}{C_2^t} = \frac{1}{C_2^t} \Big(\frac{p}{1-p}\Big)^{\sum\limits_{i=3}^n x_i} = h(p)$$ which is a function of $p$.
Hence, $T$ is NOT a sufficient statistics of $p$. However, if $T$ is the sum of ALL your samples, then you'd get a sufficient statistic.