Stability of equilibrium of a nonlinear system of ODE's

Suppose we have the nonlinear system of ODE's $$\begin{cases} \dot{x_1} = -\beta x_1 x_2 \\ \dot{x_2} = \beta x_1 x_2 - \gamma x_2 \end{cases}$$ Where we take $$\beta, \gamma > 0$$ arbitrary for now. In particular I am interested in the equilibrium point $$(x_1, x_2) = (1, 0)$$. I first linearized the system around the point $$(1, 0)$$ by using the Jacobian $$J(x_1, x_2) = \begin{pmatrix} -\beta x_2 & -\beta x_1 \\ \beta x_2 & \beta x_1 - \gamma \end{pmatrix}.$$ So the linearized system around $$(1, 0)$$ is given by $$\begin{pmatrix} \dot{x_1} \\ \dot{x_2} \end{pmatrix} = \begin{pmatrix} 0 & -\beta \\ 0 & \beta - \gamma \end{pmatrix}\begin{pmatrix} x_1 \\ x_2 \end{pmatrix}.$$ Hence, it follows we have eigenvalues $$\lambda_1 = 0$$ and $$\lambda_2 = \beta - \gamma$$. Now if $$\beta > \gamma$$ we know that the nonlinear system is unstable. However, if we let $$\beta \leq \gamma$$ we can not determine the stability of the nonlinear system by linearization.

The system seems relatively simple and I would expect the equilibrium to be stable or even asymptotically stable in the case $$\beta \leq \gamma$$, but how would one prove this when linearization fails to provide a conclusive answer? Or did I make some error in my reasoning?

Let $$u_1 = \beta x_i$$ to turn the equations into $$\left\{ \begin{array}{c} \dot{u_1} = -u_1u_2 \\ \dot{u_2} = u_1u_2-\gamma u_2\\ u_1(0) = \beta + \epsilon_1 \\ u_2(0) = \epsilon_2\end{array}\right.$$ Then get a system about $$(0,0)$$ by letting $$y_1 = u_1-\beta, y_2 = u_2$$: $$\left\{ \begin{array}{c} \dot{y_1} = -y_2(y_1+\beta) \\ \dot{y_2} =y_2(y_1+\beta) -\gamma y_2\\ y_1(0) = \epsilon_1 \\ y_2(0) = \epsilon_2\end{array}\right.$$ Then let $$z_1 = y_1+y_2, z_2=y_2$$: $$\left\{ \begin{array}{c} \dot{z_1} =-\gamma z_2\\ \dot{z_2} = z_2(z_1-z_2+\beta) \\ z_1(0) = \epsilon_1 +\epsilon_2 \equiv \epsilon_3\\ z_2(0) = \epsilon_2\end{array}\right.$$ Next, use the first equation to write $$z_2 = -\dot{z_1}/\gamma$$ and multiply the second line through by $$-\gamma$$ to get a 2nd-order ODE: $$\left\{ \begin{array}{c} \ddot{z_1} = \dot{z_1}(z_1+\dot{z_1}/\gamma + \beta) \\ z_1(0) = \epsilon_3\\ \dot{z_1}(0) = \epsilon_2\end{array}\right.$$ The given point in the original equation is stable if and only if $$z_1(0) = \dot{z_1}(0) = 0$$ is a stable fixed point in this ODE. But for $$\beta > 0$$ and ignorably small $$z_1$$ and $$\dot{z_1}$$, $$\dot{z_1}$$ grows exponentially.
So it seems the system is unstable unless $$\beta \leq 0$$.
• I doubt it. Take $0<\beta<\gamma$. Then the $x_1$-axis is the (unique) center manifold of the equilibrium $(1,0)$, consisting of equilibria, and there is a foliation by stable manifolds of equilibria close to $(1,0)$. So $(1,0)$ is stable, but not asymptotically stable. – user539887 May 6 at 21:32