Stability of nonlinear system with borderline linearization I have the following nonlinear system:
\begin{align}                                                                                                                                                               
x_1'=& x_1-x_2-x_1^3\\                                                                                                                                                      
x_2'=& x_1+x_2-x_2^3                                                                                                                                                        
\end{align}
I wish to classify the equilibrium points and determine their stability using the linearization. First, I found the equilibrium points as follows:
\begin{align}                                                                                                                                                               
\frac{\partial f_1}{\partial x_1} =& 1-3x_1^2 \\                                                                                                                            
\frac{\partial f_2}{\partial x_2} =& 1-3x_2^2 \\                                                                                                                            
\frac{\partial f_1}{\partial x_2} =& -1 \\                                                                                                                                  
\frac{\partial f_2}{\partial x_1} =& 1                                                                                                                                      
\end{align}
Therefore we have equilibria at $(1, \pm \frac{1}{\sqrt{3}})$ and $(\pm \frac{1}{\sqrt{3}}, -1)$. Thus for the jacobian:
\begin{equation}                                                                                                                                                            
J = \begin{pmatrix}\frac{\partial f_1}{\partial x_1}&\frac{\partial f_1}{\partial x_2}\\                                                                                    
                    \frac{\partial f_2}{\partial x_1}&\frac{\partial f_1}{\partial x_2}\end{pmatrix}                                                                        
\end{equation}
And the jacobian at all equilibria points is:
\begin{equation}                                                                                                                                                            
A = \begin{pmatrix}0&-1\\1&0\end{pmatrix}                                                                                                                                   
\end{equation}
Thus the eigenvalues are $\pm i$ which is purely imaginary and implies that we have a center point. However, this is a borderline case and can not be approximated by a
linearization. How do I then determine the stability of these equilibrium points then?
EDIT I am having trouble solving for the equilibrium points and I also made a mistake in the equation for $x_1'$. It should be a cubic term and not a quadratic term.
 A: To find the equilibrium points, we want to find the points where $x'_1$ and $x'_2$ are zero.
So, we want to solve:
$\tag 1 x_1-x_2-x_1^3 = 0$
$\tag 2 x_1+x_2-x_2^3=0$                                                                                                                                                     
Using $(2)$, we have $x_1 = x_2^3 - x_2$ and substituting this into equation $(1)$ yields:
$$x_2^3 - x_2 - x^2 -(x_2^3 - x_2)^3 = -x_2 (x_2^8-3 x_2^6+3 x_2^4-2 x_2^2+2) = 0$$


*

*One solution is $x_2=0$.

*The  other solutions for the eighth-order polynomial are all imaginary. I used this Wolfram Alpha solution for the roots, but you should be able to use your favorite tool for root finding.


If we substitute these back into $x_1 = x_2^3 - x_2$, we get the corresponding point as $0$. So, our critical point is $(0,0)$.
Now, you can use these two points with the Jacobian (assuming they are well behaved, do you know when you can use linearization and when you cannot) and determine how these equilibria behave.
Hint: Look at these two points in the phase portrait and your analysis should show this.

