# Where does the $\mathcal{O}$-term come from in Taylor series?

I understand Taylor series in general, but I've always been a bit uncertain about the $\mathcal{O}$-term, when I see it used in Taylor series. For example in one of my study materials I have the following:

From Taylor's theorem, for small $\varepsilon$

$$f(x^*+\varepsilon X) = f(x^*)+\varepsilon Xf'(x^*)+\mathcal{O}(\varepsilon^2)$$

Could someone give me a bit intuition, where this $\mathcal{O}$-term is coming from, because I always get a bit scared when I see it used with Taylor series. I would like to finally get this 100% x)

Well, if $f$ is analytic then $f$ has a Taylor series at $x^\ast$ of the form

$$f(x^\ast + \varepsilon X) = f(x^\ast) + f'(x^\ast) \varepsilon X + \frac{1}{2} f''(x^\ast) (\varepsilon X)^2 + \frac{1}{6} f'''(x^\ast) (\varepsilon X)^3 + \cdots$$

So let's write $g(\varepsilon)$ for all the terms after the first two:

$$f(x^\ast + \varepsilon X) = f(x^\ast) + f'(x^\ast) \varepsilon X + g(\varepsilon)$$

Now for $\varepsilon$ near zero, $\varepsilon$ is much larger than $\varepsilon^2, \varepsilon^3$, etc. So the terms in $g(x)$ often don't matter if we're taking the limit of something related to $f$ as $\varepsilon \to 0$. We want a systematic, formal way of dealing with the fact that the terms belonging to $g$ don't matter.

The way we do this is by noticing that

$$\lim_{\varepsilon \to 0} \frac{g(x)}{\varepsilon^2} = C$$

for some nonzero constant $C$. Any function $g(x)$ with this property is said to be $O(\varepsilon^2)$.

So when we do this we're forgetting exactly what function $f$ is, but remembering enough that we can calculate certain limits involving $f$.

• Excellent! Thank you very much =) – jjepsuomi Feb 2 '15 at 7:58