I understand Taylor series in general, but I've always been a bit uncertain about the $\mathcal{O}$-term, when I see it used in Taylor series. For example in one of my study materials I have the following:

From Taylor's theorem, for small $\varepsilon$

$$f(x^*+\varepsilon X) = f(x^*)+\varepsilon Xf'(x^*)+\mathcal{O}(\varepsilon^2)$$

Could someone give me a bit intuition, where this $\mathcal{O}$-term is coming from, because I always get a bit scared when I see it used with Taylor series. I would like to finally get this 100% x)

Thank you for your help!


Well, if $f$ is analytic then $f$ has a Taylor series at $x^\ast$ of the form

$$f(x^\ast + \varepsilon X) = f(x^\ast) + f'(x^\ast) \varepsilon X + \frac{1}{2} f''(x^\ast) (\varepsilon X)^2 + \frac{1}{6} f'''(x^\ast) (\varepsilon X)^3 + \cdots$$

So let's write $g(\varepsilon)$ for all the terms after the first two:

$$f(x^\ast + \varepsilon X) = f(x^\ast) + f'(x^\ast) \varepsilon X + g(\varepsilon)$$

Now for $\varepsilon$ near zero, $\varepsilon$ is much larger than $\varepsilon^2, \varepsilon^3$, etc. So the terms in $g(x)$ often don't matter if we're taking the limit of something related to $f$ as $\varepsilon \to 0$. We want a systematic, formal way of dealing with the fact that the terms belonging to $g$ don't matter.

The way we do this is by noticing that

$$\lim_{\varepsilon \to 0} \frac{g(x)}{\varepsilon^2} = C$$

for some nonzero constant $C$. Any function $g(x)$ with this property is said to be $O(\varepsilon^2)$.

So when we do this we're forgetting exactly what function $f$ is, but remembering enough that we can calculate certain limits involving $f$.

  • $\begingroup$ Excellent! Thank you very much =) $\endgroup$ – jjepsuomi Feb 2 '15 at 7:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.