Show that the origin of the following system is stable (non-linear vs linear stability)

$$\dot{x} = -2y -x^3 + x^2 y^2$$ $$\dot{y} = x - x^2 y$$

Jacobian evaluation at $(0, 0)$: $$\begin{pmatrix}0 & -2\\ 1 & 0\end{pmatrix}$$ Clearly the determinant is greater than zero but the trace is zero. Hence the linearization gives the impression of a center, which is marginally stable.

How do I show the origin is stable?

Related question: If in the case that the linearization of a system at a point is not marginally stable, is the stability of the linear system, the same as that of the non-linear system at the point (stable vs unstable)?

Near the origin we have

$$\dot{x} = -2y + O(r^3)\\ \dot{y} = x +O(r^3)$$

here $r = \sqrt{x^2+y^2}$

The stream plot for the complete system and for the approximate system NOTE

Considering the approximated system we have

$$x\dot x = -2x y\\ 2y\dot y = 2x y$$

$$\frac{d}{dt}(x^2+2y^2) = 0$$

or

$$x^2+2y^2 = C_0$$

which means that the for the approximated system the origin is a center. [![enter image description here]]

• How did you work out that $x^2 + 2y^2$ was a conserved quantity? – LMZ Jun 19 '18 at 21:02
• Why you have dropped linear terms? What theorem are you referring to when you are using this method? – Evgeny Jun 21 '18 at 5:21
• As it was shown in another answer, there is no limit cycle this close to the origin. Actually, since the original non-linear system is not structurally-stable (some of its perturbations are not equivalent to original system), numerical integration might create limit cycles which are not really present here. – Evgeny Jun 22 '18 at 15:12
• @Evgeny Interesting the formation of limit cycles due to numerical integrations. Thanks. – Cesareo Jun 22 '18 at 15:21

Consider the Lyapunov function $$V(x,y)= \frac{x^2}2+y^2.$$ Its derivative $$\dot V= x\dot x+2y\dot y=-2xy-x^4+x^3y^2+2yx-2x^2y^2 =-x^4-x^2y^2(2-x)$$ is nonpositive in $D=\{ (x,y)\in\mathbb R^2:\; x<2 \}$, thus, the origin is stable. Moreover, it is asymptotically stable, since the set $S=\{x\in D : \;\dot V(x)=0\}$ does not contain whole trajectories of the system except for the origin (see, for instance, H.Khalil, Nonlinear systems, Corollary 4.1).

The answer to the related question is no. The origin of the system $$\dot x=y,\quad \dot y=-x^3$$ is stable (the Lyapunov function is $\frac{x^4}4+\frac{y^2}2$), but the linearized system $$\dot x=y,\quad \dot y=0$$ is unstable.