1
$\begingroup$

Many times in lectures, I have seen the use of $\mathcal{O}$, especially when writing out Taylor series and so I would like to clarify if my understanding on its use is correct. I came across this question when considering a Taylor series expansion that was written as $$f(x+h)=f(x)+h \cdot f'(x)+\mathcal{O}(h^2)$$ along with the assumption that the second derivative was continuous.

This has prompted me to ask, when $\it{\text{do we}}$ and $\it{\text{can we}}$ we use this notation? My understanding is that the symbol is usually used for series expansions when the higher order terms become 'less significant' as $h \rightarrow 0$. I'm guessing this cannot be freely used and there must be some continuity assumptions applying or something?

As for the above expansion, I would argue that by the continuity of the second derivative, we can use the Lagrange form of the remainder term in Taylor series expansions and thus truncate the infinite expansion to the second order term as follows;

$$f(x+h)=f(x)+h \cdot f'(x)+\frac{h^2}{2!} \cdot f''(\xi)$$ where $\xi$ is in some interval. From this I can see why the $\mathcal{O}$ notation makes sense but I'm not sure if this is the correct explanation behind it.

$\endgroup$
  • 1
    $\begingroup$ That's exactly the right reason. Whenever the limit exists, we say a function $f$ is big $\mathcal O$ of $g$ about the point $a$ whenever $\lim_{x \to a} \frac{f}{g} $ is finite, and say $f$ is little $\mathcal o$ of $g$ whenever the limit is zero. (note that the limiting point $a$ is often given implicitly). A good place to start would be the Wikipedia page on the topic, which also gives an alternative definition more apt for computer science (where $\mathcal O$ notation is quintessential) $\endgroup$ – Brevan Ellefsen Oct 8 '18 at 22:29
1
$\begingroup$

My understanding is that the symbol is usually used for series expansions when the higher order terms become 'less significant' as $h \rightarrow 0$.

That is correct. If you want to see it through the mathematical definition of $\mathcal{O}(g(x))$. You can see this:

Well, there is actually a mathematical definition to it. You say that $f(x) \in \mathcal{O}(g(x))$ if you can prove that $\lim_{x \rightarrow 0} \frac{f(x)}{g(x)} = c$ where $c$ is not $\infty$. This is valid in your Taylor series. All terms that include $h^k$ where $k \geq 2$ are in $\mathcal{O}(h^2)$ because $\lim \frac{\alpha h^k}{h^2} = \alpha h^{k-2}$, which is $\alpha$ if $k=2$ and $0$ for $k >2$. Hence, all terms $h^k \in \mathcal{O}(h^2)$, with $k \geq 2$. This means that taylor series will be expressed as $$f(x+h)=f(x)+h \cdot f'(x)+\mathcal{O}(h^2)+\mathcal{O}(h^2)+ \ldots $$ Also, there is another Big-Oh property that uses the fact that $\mathcal{O}(h^2)+\mathcal{O}(h^2) = \mathcal{O}(h^2)$. Hence, all terms $\mathcal{O}(h^2)$ will add up to $\mathcal{O}(h^2)$. So $$f(x+h)=f(x)+h \cdot f'(x)+\underbrace{\mathcal{O}(h^2)+\mathcal{O}(h^2)+ \ldots}_{\mathcal{O}(h^2)} $$

$\endgroup$
  • $\begingroup$ The last comment about the additional property definitely helps hone in on the idea ! $\endgroup$ – user258521 Oct 8 '18 at 22:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.