This question is an extension to this previous question asked by myself:

When doing a maximum likelihood fit, we often take a ‘Gaussian approximation’. This problem works through the case of a measurement from a Poisson distribution. The Poisson distribution results in a likelihood for the average number $\mu$, given that $n$ events were observed:$L(\mu;n)=\cfrac{\mu^{n}e^{-\mu}}{n!}$

$\mu_0$ is defined to be the value of $\mu$ which gives the maximum of the likelihood, at which the likelihood has a value $L_0$.

Taking the natural logarithm of $L$ gives

$$\ln(L) = \ln(\mu^n)+\ln(e^{-\mu})-\ln(n!)=n\ln(\mu)-\mu - \ln(n!)\tag{1}$$

and taking the derivative gives


The maximum is given when $(2)$ is equal to zero; so $\mu_0$ satisfies $$\frac{n}{\mu_0}-1=0\implies \mu_0=n$$

Therefore $$\ln(L_0)=n\ln(n) - n -\ln(n!)\tag{3}$$

$\fbox{The objective in this question is to Taylor expand $\ln(L)$ to second order.}$

Performing a Taylor expansion up to the second order of $\ln(L)$ around the maximum, where $\mu_0=n$, is

$$\ln(L)\approx\ln(L_0) + \frac{\mathrm{d}\ln(L)}{\mathrm{d}\mu}\bigg|_n(\mu-n) + \frac{\mathrm{d^2}\ln(L)}{\mathrm{d}\mu^2}\bigg|_n\frac{(\mu-n)^2}{2!}$$

Taking the second derivative gives


Hence, to second order

$$\ln(L)\approx \ln(L_0) - \frac{1}{n}\frac{(\mu-n)^2}{2!}$$

$$\implies\fbox{$\color{red}{\ln(L)\approx \ln(L_0) - \frac{(\mu-n)^2}{2n}}$}\tag{?}$$

I think the final answer marked $\color{red}{\mathrm{red}}$ is wrong.

So here is my attempt:

$$\ln(L)\approx\ln(L_0) + \underbrace{\frac{\mathrm{d}\ln(L)}{\mathrm{d}\mu}\bigg|_n}_{\underbrace{=\color{blue}{\dfrac{n}{\mu}-\large{1}}}_{\Large\text{from (2)}}}\times(\mu-n) + \underbrace{\frac{\mathrm{d^2}\ln(L)}{\mathrm{d}\mu^2}\bigg|_n}_{\underbrace{=\color{#180}{-\dfrac{n}{\mu^2}\space}}_{\Large\text{from (4)}}}\times \frac{(\mu-n)^2}{2!}$$

$$\implies\ln(L)\approx\ln(L_0) + \left(\frac{n}{\mu}-1\right)\times (\mu-n) -\frac{n}{\mu^2}\times \frac{(\mu-n)^2}{2}$$

$$\implies\ln(L)\approx\ln(L_0) + \left(\frac{n-\mu}{\mu}\right)\times (\mu-n) -\frac{n(\mu-n)^2}{2\mu^2}$$



So my question is why the expression marked $\color{#F80}{\mathrm{orange}}$ is not the equal to the expression marked $\color{red}{\mathrm{red}}$?

Could someone please explain what I am doing wrong?

  • 1
    $\begingroup$ If you Taylor expand $\ln(L)$ about the point $\mu_0$ then the derivatives should be evaluated at $\mu_0$, not $\mu$. The formula you should use is: $\ln(L) = \ln(L_0) + \left.\frac{dL}{d\mu}\right|_{\mu=\mu_0}(\mu-\mu_0) + \left.\frac{d^2L}{d\mu^2}\right|_{\mu=\mu_0}\frac{(\mu-\mu_0)^2}{2}$. $\endgroup$
    – Winther
    Nov 29, 2015 at 23:20
  • $\begingroup$ @Winther Thanks for your reply. This is a word for word copy of the proof that I have been given, if it is indeed wrong then this needs addressing and that is the purpose of this post and $\mu_0=n$. $\endgroup$
    – BLAZE
    Nov 29, 2015 at 23:26
  • 1
    $\begingroup$ I'm trying to address the mistake in your try. The red answer is correct and the reason your try is wrong is that you use $\frac{dL}{d\mu} = \frac{n}{\mu}-1$ instead of $\left.\frac{dL}{d\mu}\right|_{\mu=\mu_0 = n} = \frac{n}{n}-1 = 0$ and similar for the second derivatives in the formula for the second order Taylor polynomial. $\endgroup$
    – Winther
    Nov 29, 2015 at 23:32
  • $\begingroup$ @Winther Okay, very good; now we're getting somewhere. It seems the cause of my misunderstanding is due to not knowing what the notation $\frac{\mathrm{d}\ln(L)}{\mathrm{d}\mu}\bigg|_n$ means. Could you please elaborate on this my making an answer to this question? I feel I owe you some rep for this, thank you. $\endgroup$
    – BLAZE
    Nov 29, 2015 at 23:40

1 Answer 1


The red answer is correct. The mistake in the orange answer is simply forgetting to evaluate the derivatives at the point $\mu = \mu_0 = n$ and instead using the general variable $\mu$ in these expressions.

The second order Taylor polynomial of $\ln(L)$ about $\mu=n$ is given by

$$\ln(L_0) + \color{red}{\left.\frac{d\ln L}{d\mu}\right|_{\mu=n}}(\mu-n) + \color{blue}{\left.\frac{d^2\ln L}{d\mu^2}\right|_{\mu=n}}\frac{(\mu-n)^2}{2}\\= \ln(L_0) + \color{red}{0}\cdot (\mu-n) + \left(\color{blue}{-\frac{1}{n}}\right)\frac{(\mu-n)^2}{2} = \ln(L_0) - \frac{(\mu-n)^2}{2n}$$


$$\left.\frac{d\ln L}{d\mu}\right|_{\mu=n} = \left(\frac{n}{\mu}-1\right)_{\mu=n} = 0,~~~~~~~~~\left.\frac{d^2\ln L}{d\mu^2}\right|_{\mu=n} = \left(-\frac{n}{\mu^2}\right)_{\mu=n} = -\frac{1}{n}$$

  • $\begingroup$ Perfect answer, thanks very much. I think my confusion was due to not knowing what the vertical bar (or whatever it's called) stands for. It seems to have different meanings: like in partial derivatives to take the derivative while holding one of the variables constant; $\cfrac{\partial u(x,y)}{\partial x}\bigg{|}_y$ and this $\cfrac{\mathrm{d}\ln(L)}{\mathrm{d}\mu}\bigg|_n$ to evaluate at $\mu=n$. $\endgroup$
    – BLAZE
    Nov 30, 2015 at 8:53

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .