# Why does each successive term in a Taylor series need to be much less than the previous term?

This is an extension to this previous question for this original question asked by myself:

When doing a maximum likelihood fit, we often take a ‘Gaussian approximation’. This problem works through the case of a measurement from a Poisson distribution. The Poisson distribution results in a likelihood for the average number $\mu$, given that $n$ events were observed:$L(\mu;n)=\cfrac{\mu^{n}e^{-\mu}}{n!}$.

We need to write this in the form $$L(\mu;n)\approx L_0 e^{-\frac{(\mu-n)^2}{2n}}$$ $\mu_0$ is defined to be the value of $\mu$ which gives the maximum of the likelihood, at which the likelihood has a value $L_0$.

Taking the natural logarithm of $L$ gives

$$\ln(L) = \ln(\mu^n)+\ln(e^{-\mu})-\ln(n!)=n\ln(\mu)-\mu - \ln(n!)$$

and taking the derivative gives

$$\frac{\mathrm{d}\ln(L)}{\mathrm{d}\mu}=\color{maroon}{\frac{n}{\mu}-1}$$

The maximum is given when $(2)$ is equal to zero; so $\mu_0$ satisfies $$\frac{n}{\mu_0}-1=0\implies \mu_0=n$$

Therefore $$\ln(L_0)=n\ln(n) - n -\ln(n!)$$

Performing a Taylor expansion up to the second order of $\ln(L)$ around the maximum, where $\mu_0=n$, is

$$\ln(L)\approx\ln(L_0) + \frac{\mathrm{d}\ln(L)}{\mathrm{d}\mu}\bigg|_{\mu=n}(\mu-n) + \frac{\mathrm{d^2}\ln(L)}{\mathrm{d}\mu^2}\bigg|_{\mu=n}\frac{(\mu-n)^2}{2!}$$

Taking the second derivative gives

$$\frac{\mathrm{d^2}\ln(L)}{\mathrm{d}\mu^2}=\color{#180}{-\frac{n}{\mu^2}}\tag{1}$$

Hence, to second order

$$\ln(L)\approx \ln(L_0) \color{blue}{- \frac{(\mu-n)^2}{2n}}$$

$$\implies \frac{L}{L_0}\approx e^{-\frac{(\mu-n)^2}{2n}}\implies \fbox{\color{red}{L(\mu;n)\approx L_0 e^{-\frac{(\mu-n)^2}{2n}}}}\tag{2}$$

The Gaussian approximation therefore comes down to neglecting the third order and higher terms. The first neglected term in the Taylor expansion is third order.

Evaluating this derivative gives $$\frac{\mathrm{d^3}\ln(L)}{\mathrm{d}\mu^3}=\color{#F80}{\frac{2n}{\mu^3}}$$ Hence, the first neglected term is $$\frac{\mathrm{d^3}\ln(L)}{\mathrm{d}\mu^3}\bigg|_{\mu=n}\frac{(\mu-n)^3}{3!}=\frac{2}{n^2}\frac{(\mu-n)^3}{6}=\frac{(\mu-n)^3}{3n^2}$$

For the approximation to be valid, this must be much smaller than the second order term, so

$$\fbox{\color{purple}{\frac{(\mu-n)^3}{3n^2}\ll\frac{(\mu-n)^2}{2n}}}\tag{???}$$

I have two simple questions regarding the expression marked $\color{purple}{\mathrm{purple}}$ since the second order term shown earlier in $\color{blue}{\mathrm{blue}}$ is negative: $\color{blue}{-\cfrac{(\mu-n)^2}{2n}}$. So the $\color{purple}{\mathrm{purple}}$ inequality cannot possibly be true.

However, assuming for my first question that we are only concerned with the magnitude (neglecting the negative sign); Why does the third order term need to be $\ll$ second order term? On what grounds is this necessary?

Thanks.

The first neglected term should be smaller in magnitude, so that $\big|\frac{(\mu-n)^3}{3n^2}\big| \ll \big|\frac{(\mu-n)^2}{2n}\big|$ or $|\mu-n| \ll \frac32 n$.
• Excellent, that has answered my first question; Now can you please explain to me why the third order term needs to be $\ll$ second order term? Thank you. – BLAZE Dec 5 '15 at 10:21
If a function is bounded on an interval or a set and has a Taylor expansion there, if we were not sure that new monomial terms had coefficients smaller than the previous at least such that $c_{k+1} \leq k xc_k$ for the $x$ in the area of convergence then the value could increase without bounds but that can not be because the function does not. Which $k$ would be sufficient for the series to diverge?