# How do you Taylor expand the log likelihood function of the Poisson distribution?

This question is an extension to this previous question asked by myself:

When doing a maximum likelihood fit, we often take a ‘Gaussian approximation’. This problem works through the case of a measurement from a Poisson distribution. The Poisson distribution results in a likelihood for the average number $\mu$, given that $n$ events were observed:$L(\mu;n)=\cfrac{\mu^{n}e^{-\mu}}{n!}$

$\mu_0$ is defined to be the value of $\mu$ which gives the maximum of the likelihood, at which the likelihood has a value $L_0$.

Taking the natural logarithm of $L$ gives

$$\ln(L) = \ln(\mu^n)+\ln(e^{-\mu})-\ln(n!)=n\ln(\mu)-\mu - \ln(n!)\tag{1}$$

and taking the derivative gives

$$\frac{\mathrm{d}\ln(L)}{\mathrm{d}\mu}=\color{blue}{\frac{n}{\mu}-1}\tag{2}$$

The maximum is given when $(2)$ is equal to zero; so $\mu_0$ satisfies $$\frac{n}{\mu_0}-1=0\implies \mu_0=n$$

Therefore $$\ln(L_0)=n\ln(n) - n -\ln(n!)\tag{3}$$

$\fbox{The objective in this question is to Taylor expand$\ln(L)$to second order.}$

Performing a Taylor expansion up to the second order of $\ln(L)$ around the maximum, where $\mu_0=n$, is

$$\ln(L)\approx\ln(L_0) + \frac{\mathrm{d}\ln(L)}{\mathrm{d}\mu}\bigg|_n(\mu-n) + \frac{\mathrm{d^2}\ln(L)}{\mathrm{d}\mu^2}\bigg|_n\frac{(\mu-n)^2}{2!}$$

Taking the second derivative gives

$$\frac{\mathrm{d^2}\ln(L)}{\mathrm{d}\mu^2}=\color{#180}{-\frac{n}{\mu^2}}\tag{4}$$

Hence, to second order

$$\ln(L)\approx \ln(L_0) - \frac{1}{n}\frac{(\mu-n)^2}{2!}$$

$$\implies\fbox{\color{red}{\ln(L)\approx \ln(L_0) - \frac{(\mu-n)^2}{2n}}}\tag{?}$$

I think the final answer marked $\color{red}{\mathrm{red}}$ is wrong.

So here is my attempt:

$$\ln(L)\approx\ln(L_0) + \underbrace{\frac{\mathrm{d}\ln(L)}{\mathrm{d}\mu}\bigg|_n}_{\underbrace{=\color{blue}{\dfrac{n}{\mu}-\large{1}}}_{\Large\text{from (2)}}}\times(\mu-n) + \underbrace{\frac{\mathrm{d^2}\ln(L)}{\mathrm{d}\mu^2}\bigg|_n}_{\underbrace{=\color{#180}{-\dfrac{n}{\mu^2}\space}}_{\Large\text{from (4)}}}\times \frac{(\mu-n)^2}{2!}$$

$$\implies\ln(L)\approx\ln(L_0) + \left(\frac{n}{\mu}-1\right)\times (\mu-n) -\frac{n}{\mu^2}\times \frac{(\mu-n)^2}{2}$$

$$\implies\ln(L)\approx\ln(L_0) + \left(\frac{n-\mu}{\mu}\right)\times (\mu-n) -\frac{n(\mu-n)^2}{2\mu^2}$$

$$\implies\ln(L)\approx\ln(L_0)-\frac{(\mu-n)^2}{\mu}-\frac{n(\mu-n)^2}{2\mu^2}$$

$$\implies\color{#F80}{\ln(L)\approx\ln(L_0)-\frac{(\mu-n)^2}{\mu}\left(\frac{n}{2\mu}+1\right)}$$

So my question is why the expression marked $\color{#F80}{\mathrm{orange}}$ is not the equal to the expression marked $\color{red}{\mathrm{red}}$?

Could someone please explain what I am doing wrong?

• If you Taylor expand $\ln(L)$ about the point $\mu_0$ then the derivatives should be evaluated at $\mu_0$, not $\mu$. The formula you should use is: $\ln(L) = \ln(L_0) + \left.\frac{dL}{d\mu}\right|_{\mu=\mu_0}(\mu-\mu_0) + \left.\frac{d^2L}{d\mu^2}\right|_{\mu=\mu_0}\frac{(\mu-\mu_0)^2}{2}$. Nov 29, 2015 at 23:20
• @Winther Thanks for your reply. This is a word for word copy of the proof that I have been given, if it is indeed wrong then this needs addressing and that is the purpose of this post and $\mu_0=n$. Nov 29, 2015 at 23:26
• I'm trying to address the mistake in your try. The red answer is correct and the reason your try is wrong is that you use $\frac{dL}{d\mu} = \frac{n}{\mu}-1$ instead of $\left.\frac{dL}{d\mu}\right|_{\mu=\mu_0 = n} = \frac{n}{n}-1 = 0$ and similar for the second derivatives in the formula for the second order Taylor polynomial. Nov 29, 2015 at 23:32
• @Winther Okay, very good; now we're getting somewhere. It seems the cause of my misunderstanding is due to not knowing what the notation $\frac{\mathrm{d}\ln(L)}{\mathrm{d}\mu}\bigg|_n$ means. Could you please elaborate on this my making an answer to this question? I feel I owe you some rep for this, thank you. Nov 29, 2015 at 23:40

The red answer is correct. The mistake in the orange answer is simply forgetting to evaluate the derivatives at the point $\mu = \mu_0 = n$ and instead using the general variable $\mu$ in these expressions.
The second order Taylor polynomial of $\ln(L)$ about $\mu=n$ is given by
$$\ln(L_0) + \color{red}{\left.\frac{d\ln L}{d\mu}\right|_{\mu=n}}(\mu-n) + \color{blue}{\left.\frac{d^2\ln L}{d\mu^2}\right|_{\mu=n}}\frac{(\mu-n)^2}{2}\\= \ln(L_0) + \color{red}{0}\cdot (\mu-n) + \left(\color{blue}{-\frac{1}{n}}\right)\frac{(\mu-n)^2}{2} = \ln(L_0) - \frac{(\mu-n)^2}{2n}$$
$$\left.\frac{d\ln L}{d\mu}\right|_{\mu=n} = \left(\frac{n}{\mu}-1\right)_{\mu=n} = 0,~~~~~~~~~\left.\frac{d^2\ln L}{d\mu^2}\right|_{\mu=n} = \left(-\frac{n}{\mu^2}\right)_{\mu=n} = -\frac{1}{n}$$
• Perfect answer, thanks very much. I think my confusion was due to not knowing what the vertical bar (or whatever it's called) stands for. It seems to have different meanings: like in partial derivatives to take the derivative while holding one of the variables constant; $\cfrac{\partial u(x,y)}{\partial x}\bigg{|}_y$ and this $\cfrac{\mathrm{d}\ln(L)}{\mathrm{d}\mu}\bigg|_n$ to evaluate at $\mu=n$. Nov 30, 2015 at 8:53