Confusion over big O notation in numerical analysis I need help with understanding the big O notation used in numerical analysis. 
so $f(h) = O(h) $ if $ |f(h)|\leq k|h|$ for some constant $k$. 
Here, is $k$ a positive real number? 
Does the equality needs to hold for all value of $h$ in the domain of $f$ and for a $k$ instead of all $k>0$ ?
Also, for the big O notation used on sequences,
suppose $\{b_{n}\}$ converges to $0$ and $\{a_{n}\}$ converges to $\alpha$, if there is a constant $K$ such that $|a_{n}-\alpha|\leq K|b_{n}|$, then $a_{n}$ converges to $\alpha$ with the rate of convergence $O(b_{n})$.
Most of the time, it is then written as $a_{n} = \alpha + O(b_{n})$. Why is this written as such? What does the equality sign here actually means?
Also, similar notation is used for functions.
What does it mean that a function converges?
 A: This is one of those topics that people argue about, no less because what some people see as a horrible "abuse of notation" others see as a useful convention,
and because the exact meaning of a particular notation may depend on who is using it, when they are using it, and possibly even what particular functions are plugged into it.
When people write that $f(h) = O(h) $ if $ |f(h)|\leq k|h|$,
they do not really mean "equal" (because usually $f(h)$ is a particular function, and $O(h)$ never is),
they only need the inequality to be true for at least one value of $k$,
and they do not need it to be true for all values of $h$.
For numerical analysis, especially when the variable is named $h$
(which usually stands for the length of a step in a numeric integration
or some other relatively small number), 
usually you only need $|f(h)|\leq k|h|$ to be true for small values of $h$,
for example if you can find some number $\delta$, and show that
$|f(h)|\leq k|h|$ whenever $|h| < \delta$, that's good enough.
And after the first few exercises you can usually show that a
function is $O(g(h))$ without actually finding exactly how small
$h$ needs to be. (That is, you don't need to actually know the number
$\delta$ that I just mentioned.)
But for computational complexity (how long a program takes),
when someone mentions $O(g(n))$ they usually mean what happens
for very large values of $n$. Formally, there has to be some number $M$
and some number $k$ such that $|f(n)|\leq k|g(n)|$ whenever $n > M$.
It doesn't sound like this is what you're dealing with, but some day
you may be learning about complexity or some other topic and then it will be important to know whether the particular big-O notation you are seeing at that particular time is for very small values of the variable or for very large values.
Some people say you should always write $f(h) \in O(h) $
and never $f(h) = O(h),$ because $O(h)$ is really a class 
(or set) of functions and $f$ is only a member of it.
But then you will see notation like $a_n = \alpha + O(n^3)$,
which may mean only that there is some function $g(n)$ in the class $O(n^3)$
such that $a_n = \alpha + g(n)$.
Similarly,
you may see something like $f(x) = 1 + x + O(x^2)$ for small values of $x$,
which means there is some function $g(x)$ in the class $O(x^2)$
such that $f(x) = 1 + x + g(x)$.
And in this last case ($g(x)$ in the class $O(x^2)$ for small $x$),
$g(x)$ could be $x^2$ or $\frac12x^2$ or $2x^2 + 3x^3$ or
$\frac12x^2+\frac16x^3+\frac1{24}x^4+\cdots$ or any other function that
there's some value of $k$ so that $|g(x)| \leq kx^2$ when $x$ is small enough.
A: An important point is that if we say $f(x)=O(g(x))$ as $x\to 0,$ we mean there exists $C>0$ and $r>0$ such that for all $x\in (-r,r)\cap dom (f)$ we have $|f(x)|\leq C|g(x)|.$ 
It doesn't require that $|f|\leq C|g|$ for all $x.$
Similarly $f(x)=O(g(x))$ as $x\to \infty$ means there exists $C>0, r>0$ such that for all $x\in (r,\infty)\cap dom (f)$ we have $|f(x)|\leq  C|g(x)|.$ 
For example $2x^2+20x =O(x^2)$ as $x\to \infty.$ 
And you must specify what $x$ is "tending to" , just as you must do so when referring to a limit.
