Using Bendixson-Dulac to show no cyle exists for system I have the following system $x'=f(x)$:
\begin{align}
x_1'=& 1+x_1x_2 \\
x_2'=& 2x_1^2+x_1^2x_2^2
\end{align}
I want to show there is no cycle (maybe I am wrong)
Maybe I am using the wrong theorem, but I plugged in a general $\alpha(x,y)$. To try to find $div(\alpha f)$ such that $div(\alpha f)$ is always less than or greater than zero. However, I didn't really get anywhere.
How can I find out of there is a cycle?
 A: My calculations show that you can take $\alpha(x_1, x_2)$ equal to $x_2$ and apply Bendixson-Dulac to $\alpha\cdot f$.
EDIT So, with a remark from topic starter I have to reformulate my answer. As it is proved below, it can be shown that this system has no closed trajectories that cross $x_1=0$ axis. Trajectories that go around origin are also excluded. Take bounded domain $D$ which boundary is hypothetical closed trajectory. It's not hard to construct simply-connected domain $G$ that contains closure of $D$ and that doesn't contain origin. You can use the offered function to apply Bendixson-Dulac for domain $G$ and get a contradiction. So, this shows that this system has no closed trajectories at all.
As for your second question... Suppose you know where are located all equilibria states and suppose you have two closed simple smooth (or piecewise smooth) curves $\gamma_1$, $\gamma_2$ with following properties: (1) one of the curves lies inside the bounded part of plain which boundary is another curve, (2) tangent vectors along curves are transversal to the vector field (it's called "curves without contact"), (3) no equilibria states in the annulus between $\gamma_1$, $\gamma_2$. The 4th condition would be: vector field points "into" the annulus or "out" of it simultaneously on both curves. These conditions may help you to find out that there is a limit cycle in the annulus. That's because the limit cycles and equilibria states are the only possible examples of $\alpha$- and $\omega$-sets in bounded domains at plane.
