1
$\begingroup$

I am trying to calculate the Fisher Information for a $Bin(n,p)$ with the following parametrization:

$$\theta=\text{log}\frac{p}{1-p}$$ which implies: $$p=\frac{e^\theta}{1+e^\theta}$$

I tried two methods but got two different answers:

$$f_\theta(x)={n\choose k}p^{x}(1-p)^{n-x}$$

In canonical form:

$$f_\theta(x)=e^{log{n\choose k}+x\theta+nlog(\frac{1}{1+e^\theta})}$$

then the fisher information is $$\frac{d^{2}(-n*log\frac{1}{1+e^\theta})}{d\theta^2}=\frac{-ne^\theta}{(1+e^\theta)^2}$$

When I calculate it directly(not using canonical form i.e taking the second derivative and taking the negative expected value) I get:

$$log f_\theta(x)=log {n\choose x}+x*log(\frac{e^\theta}{1+e^\theta})+(n-x)log(\frac{1}{1+e^\theta})$$ $$\frac{\partial}{\partial \theta}log f_\theta(x)=\frac{x(1-e^\theta)}{e^\theta}-(n-x)(1-e^\theta)$$

$$\frac{\partial^2}{\partial \theta^2}log f_\theta(x)=(n-x)e^\theta -\frac{x}{e^\theta}$$

$$E[-\frac{\partial^2}{\partial \theta^2}log f_\theta(x)]=\frac{n(1-e^\theta)}{1+e^\theta}$$ Which one is correct?

$\endgroup$
8
  • $\begingroup$ What do you mean by calculation directly? The first one is correct $\endgroup$
    – NCh
    Commented Jan 28, 2020 at 16:51
  • $\begingroup$ Taking the second derivative and taking the negative expected value $\endgroup$
    – nvm
    Commented Jan 28, 2020 at 16:53
  • $\begingroup$ The second derivative with respect to what? $\endgroup$
    – NCh
    Commented Jan 28, 2020 at 16:54
  • $\begingroup$ I am taking the derivative with respect to $\theta$ $\endgroup$
    – nvm
    Commented Jan 28, 2020 at 16:56
  • $\begingroup$ There cannot be any difference since you do the same twice. You can add to the question the mentioned direct calculations in order that we can find error. $\endgroup$
    – NCh
    Commented Jan 28, 2020 at 16:59

1 Answer 1

1
$\begingroup$

Use $f(g(x))'=f'(g(x))\cdot g'(x)$. So $$ \frac{\partial}{\partial \theta}\left(x\log(\frac{e^\theta}{1+e^\theta})\right) = x \frac{1+e^\theta}{e^\theta}\cdot \frac{\partial}{\partial \theta}\left(\frac{e^\theta}{1+e^\theta} \right) = x \frac{1+e^\theta}{e^\theta}\cdot \frac{\partial}{\partial \theta}\left(1-\frac{1}{1+e^\theta} \right) $$ $$ = x \frac{1+e^\theta}{e^\theta}\cdot \frac{1}{(1+e^\theta)^2} \cdot e^\theta = x\frac{1}{1+e^\theta}. $$ And the second derivative is $$ \frac{\partial}{\partial \theta}\left(x\frac{1}{1+e^\theta}\right) =- x \frac{1}{(1+e^\theta)^2} \cdot e^\theta. $$

$\endgroup$
1
  • $\begingroup$ you are correct I made a mistake and kept the derivation. I get the same answer now. Math is not broken!. Thank you $\endgroup$
    – nvm
    Commented Jan 28, 2020 at 17:32

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .