Stability equilibrium point of non linear system with nonpositive eigenvalues. We have the following non linear system:\begin{equation}
\dot{X}=f(X),
\end{equation}
and lets suppose that $f(X_0)=0$. We also have that $\lambda_1,\dots,\lambda_n$ are the eigenvales of $D_f(X_0)$ and there are $Re(\lambda_1),\dots,Re(\lambda_k)=0$ for some $k<n$ and $Re(\lambda_{k+1}),\dots,Re(\lambda_n)<0$ .
What can we say about the stability of the system? Can we say is stable because it is not unstable?
 A: You cannot determine the stability in general here. You need to consider the Center Manifold theorem. By changing basis you can reduce your system to 
$$\dot{X}_c = A_c X_c + g_c(X_c,X_s)$$
$$\dot{X}_s = A_s X_s + g_s(X_c,X_s)$$
where $A_s$ has $n-k$ eigenvalues with negative real parts and $A_c$ has $k$ eigenvalues with zero real parts. Furthermore
$$g_c : \mathbb{R}^n \mapsto E_c$$
$$g_c : \mathbb{R}^n \mapsto E_s$$
Where $E_c$ and $E_s$ are your center and stable eigenspaces respectively and $g_i$ contains no constant or linear terms. In the case of no eigenvalues with positive real parts, it can be shown that all solutions are attracted to a center manifold at an exponential rate, so stability is determined by what happens on the center manifold. The center manifold theorem states there exists a neighborhood $U\subset E_c$ of $X_c^* = 0$ and a function $h\in C^{r-1}$ (when $f\in C^r$)  such that
$$h  :U\mapsto E_s$$
such that $h$ contains no constant or linear terms and defines a center manifold
$$W_c := \{ X=(X_c,X_s) \ | \ X_c\in  U, h(X_c) = X_s \} $$
and your system is then reduced to one on the (non necessarily unique) center manifold
$$\dot{X}_c = A_cX_c + g_c(X_c, h(X_c))$$
For example take
$$\dot{x} = xy \\ \dot{y} = -y \pm x^2$$
Then by the center manifold theorem we can take 
$$y(x) = h(x) = a_1x^2 + a_2x^3 + ...$$
hence 
$$\dot{y} = h'(x)\dot{x} = [2a_1x + 3a_2x^2+ ...][x][a_1x^2 + a_2x^3 + ...] \\
= -[a_1x^2 + a_2x^3 + ...] \pm x^2$$
comparing the coefficients of $x^n$ we find that $a_1 = \mp 1$, $a_2 = 0$ and $a_3 = -2$ and so on. Finally
$$\dot{x} = xy = \mp x^3 -2x^5 + \mathcal{O}(x^7)$$
Which is stable for $-x^3$ and unstable for $+x^3$
A: Here is a very simple example, a system on $\Bbb R^2$ which has a single equilibrium, a center, at $(0,0)$; the system is however unstable.
We let 
$\dot x = y + x^3, \tag 1$
$\dot y = -x + y^3; \tag 2$
then
$\dot x = \dot y = 0 \Longrightarrow y = -x^3, \; x = y^3, \tag 3$
whence
$x = y^3 = (-x^3)^3 = -x^9, \tag 4$
$x^9 + x = 0, \tag 5$
$x(x^8 + 1) = 0; \tag 6$
only real solution is evidently
$x = 0, \tag 7$
since $x^8 + 1 > 0$ for all real $x$; thus, the only equilibrium of (1)-(2) is 
$(x, y) = (x, -x^3) = (0, 0); \tag 8$
the Jacobian matrix of (1)-(2) is
$J(x, y) = \begin{bmatrix} 3x^2 & 1 \\ -1 & 3y^2 \end{bmatrix}, \tag 9$
whence
$J(0, 0) = \begin{bmatrix} 0 & 1 \\ -1 & 0 \end{bmatrix}; \tag{10}$
the characteristic polynomial of $J(0,0)$ is
$\det(J(0,0) - \lambda I) = \det \left (\begin{bmatrix} -\lambda & 1 \\ -1 & -\lambda \end{bmatrix} \right ) = \lambda^2 + 1; \tag{11}$
thus the eigenvalues of $J(0, 0)$ are $\pm i$; $(0,0)$ is a center; but with
$r^2 = x^2 + y^2, \tag{12}$
we have
$\dfrac{d}{dt}(r^2) = 2r\dot r = 2x \dot x + 2y\dot y$
$= 2xy + 2x^4 - 2yx + 2y^4 = 2(x^4 + y^4) > 0, \forall (x, y) \ne (0, 0); \tag{13}$
thus $r^2$, and hence $r$, is strictly monotonically increasing for all $r \ne 0$; the equilibrium point $(0,0)$, albeit a center, is unstable.
