Composition of functions with little o Let's suppose that:
$f(x)=h(x)+\frac{1}{x}o(1)$ when $x\to\infty$ where $h(x)=o(1)$ and $xh^2(x)\to\infty$ when $x\to\infty$
Also, suppose that $g(t)=at+bt^2+t^2o(1)$ when $t\to 0$
Prove that, when $x\to\infty$ $$g(f(x))=ah(x)+bh^2(x)+h^2(x)o(1)$$
This is what I did:
$g(f(x))=g\left(h(x)+\frac{1}{x}o(1)\right)$ when $\left(h(x)+\frac{1}{x}o(1)\right)\to0$
$g(h(x)+\frac{1}{x}o(1))=a(h(x)+\frac{1}{x}o(1))+b({h(x)+\frac{1}{x}o(1)})^2+(h(x)+\frac{1}{x}o(1))^2o(1)$
$=ah(x)+a\frac{1}{x}o(1)+bh^2(x)+2bh(x)\frac{1}{x}o(1)+b\frac{1}{x^2}o(1)+h^2(x)o(1)+\frac{2}{x}h(x)o(1)+\frac{1}{x^2}o(1)=
ah(x)+\frac1xo(1) + bh^2(x)+h(x)\frac1xo(1)+\frac1{x^2}o(1)+h^2(x)o(1)+h(x)\frac1xo(1)+\frac{1}{x^2}o(1)= ah(x)+bh^2(x)+h^2(x)o(1)+\frac1xo(1)+h(x)\frac1xo(1)+\frac1{x^2}o(1)$
I don't know how to proceed (mostly beacause I don't know the little o properties), can I get some help?
Note: I've been told that what I have to prove is that $\frac1x=h^2(x)o(1)$, why is that?
2nd note: Can I just say that the last terms are 0 when $x\to\infty$?
 A: By definition small $o$ with $x\to x_0$ we have
$$o(f) = \left\lbrace g: \exists \delta >0, \exists \varepsilon (x), \lim\limits_{x\to x_0} \varepsilon (x)=0,\ \forall x \in U(x_0,\delta), g(x)= \varepsilon (x) \cdot f(x) \right\rbrace$$
Where $U(x_0,\delta)$ is some $\delta$ neigbourhood of $x_0$.
From this definition we have, for example, following properties:

*

*$C \cdot o(f)=o(f)$, $C \ne 0$ constant.

*$o\left(o(f)\right) = o(f)$
Your last question in "Note" equality can be rewrite as $\frac1x=h^2(x)\varepsilon (x)$, where $\lim\limits_{x\to \infty} \varepsilon (x)=0$. I assume you mean $x\to \infty$.
Now about main question. In your last line, if we take out members $ah(x)+bh^2(x)$, then it will be
$$\frac1xo(1) + h(x)\frac1xo(1)+\frac1{x^2}o(1)+h^2(x)o(1)+h(x)\frac1xo(1)+\frac{1}{x^2}o(1)$$
Taking $h^2(x)$ out of brackets, you need conditions by which will be limit zero for following members $\frac{1}{h^2x},\frac{1}{hx},\frac{1}{x^2h^2}$. First one you have from "Note" $\frac{1}{h^2x}=o(1)$. For second one $\frac{1}{hx}=\frac{h}{h^2x}$ and we use $h=o(1)$. 3d one I hope you can manage?
