How do you Taylor expand the log likelihood function of the Poisson distribution?

Question

This question is an extension to this previous question asked by myself:

When doing a maximum likelihood fit, we often take a ‘Gaussian approximation’. This problem works through the case of a measurement from a Poisson distribution. The Poisson distribution results in a likelihood for the average number $\mu$, given that $n$ events were observed:$L(\mu;n)=\cfrac{\mu^{n}e^{-\mu}}{n!}$

$\mu_0$ is defined to be the value of $\mu$ which gives the maximum of the likelihood, at which the likelihood has a value $L_0$.

---

Taking the natural logarithm of $L$ gives

$$\ln(L) = \ln(\mu^n)+\ln(e^{-\mu})-\ln(n!)=n\ln(\mu)-\mu - \ln(n!)\tag{1}$$

and taking the derivative gives

$$\frac{\mathrm{d}\ln(L)}{\mathrm{d}\mu}=\color{blue}{\frac{n}{\mu}-1}\tag{2}$$

The maximum is given when $(2)$ is equal to zero; so $\mu_0$ satisfies $$\frac{n}{\mu_0}-1=0\implies \mu_0=n$$

Therefore $$\ln(L_0)=n\ln(n) - n -\ln(n!)\tag{3}$$

$\fbox{The objective in this question is to Taylor expand $\ln(L)$ to second order.}$

Performing a Taylor expansion up to the second order of $\ln(L)$ around the maximum, where $\mu_0=n$, is

$$\ln(L)\approx\ln(L_0) + \frac{\mathrm{d}\ln(L)}{\mathrm{d}\mu}\bigg|_n(\mu-n) + \frac{\mathrm{d^2}\ln(L)}{\mathrm{d}\mu^2}\bigg|_n\frac{(\mu-n)^2}{2!}$$

Taking the second derivative gives

$$\frac{\mathrm{d^2}\ln(L)}{\mathrm{d}\mu^2}=\color{#180}{-\frac{n}{\mu^2}}\tag{4}$$

Hence, to second order

$$\ln(L)\approx \ln(L_0) - \frac{1}{n}\frac{(\mu-n)^2}{2!}$$

$$\implies\fbox{$\color{red}{\ln(L)\approx \ln(L_0) - \frac{(\mu-n)^2}{2n}}$}\tag{?}$$

I think the final answer marked $\color{red}{\mathrm{red}}$ is wrong.

---

So here is my attempt:

$$\ln(L)\approx\ln(L_0) + \underbrace{\frac{\mathrm{d}\ln(L)}{\mathrm{d}\mu}\bigg|_n}_{\underbrace{=\color{blue}{\dfrac{n}{\mu}-\large{1}}}_{\Large\text{from (2)}}}\times(\mu-n) + \underbrace{\frac{\mathrm{d^2}\ln(L)}{\mathrm{d}\mu^2}\bigg|_n}_{\underbrace{=\color{#180}{-\dfrac{n}{\mu^2}\space}}_{\Large\text{from (4)}}}\times \frac{(\mu-n)^2}{2!}$$

$$\implies\ln(L)\approx\ln(L_0) + \left(\frac{n}{\mu}-1\right)\times (\mu-n) -\frac{n}{\mu^2}\times \frac{(\mu-n)^2}{2}$$

$$\implies\ln(L)\approx\ln(L_0) + \left(\frac{n-\mu}{\mu}\right)\times (\mu-n) -\frac{n(\mu-n)^2}{2\mu^2}$$

$$\implies\ln(L)\approx\ln(L_0)-\frac{(\mu-n)^2}{\mu}-\frac{n(\mu-n)^2}{2\mu^2}$$

$$\implies\color{#F80}{\ln(L)\approx\ln(L_0)-\frac{(\mu-n)^2}{\mu}\left(\frac{n}{2\mu}+1\right)}$$

So my question is why the expression marked $\color{#F80}{\mathrm{orange}}$ is not the equal to the expression marked $\color{red}{\mathrm{red}}$?

Could someone please explain what I am doing wrong?

Answer 1

The red answer is correct. The mistake in the orange answer is simply forgetting to evaluate the derivatives at the point $\mu = \mu_0 = n$ and instead using the general variable $\mu$ in these expressions.

The second order Taylor polynomial of $\ln(L)$ about $\mu=n$ is given by

$$\ln(L_0) + \color{red}{\left.\frac{d\ln L}{d\mu}\right|_{\mu=n}}(\mu-n) + \color{blue}{\left.\frac{d^2\ln L}{d\mu^2}\right|_{\mu=n}}\frac{(\mu-n)^2}{2}\\= \ln(L_0) + \color{red}{0}\cdot (\mu-n) + \left(\color{blue}{-\frac{1}{n}}\right)\frac{(\mu-n)^2}{2} = \ln(L_0) - \frac{(\mu-n)^2}{2n}$$

since

$$\left.\frac{d\ln L}{d\mu}\right|_{\mu=n} = \left(\frac{n}{\mu}-1\right)_{\mu=n} = 0,~~~~~~~~~\left.\frac{d^2\ln L}{d\mu^2}\right|_{\mu=n} = \left(-\frac{n}{\mu^2}\right)_{\mu=n} = -\frac{1}{n}$$