# Unstable fixed points

When considering the system

$$\begin{cases} x' = (A-By)x \\ y' = (C-Dx)y, & \end{cases}$$

($$A,B,C,D > 0$$)

I am trying to understand how to tell that the fixed points $$(0,0)$$ and $$(C/D,A/B)$$ are unstable. If I'm not mistaken, Lyapunov functions can be used to show that a point is stable but not that it is unstable. I was thinking of finding the eigenvalues (if we consider the system as $$\bf{x'} = Ax$$), but the eigenvalues will involve $$x$$ and $$y$$ and that hasn't happened before as I've been solving exercises, so I'm not sure if that is the way to go...

• Incidentally, there are Lyapunov functions (sort of) that can be used to prove that an equilibrium is unstable; see, e.g., Chetaev function. Commented Mar 14, 2019 at 8:07

The eigenvalues will stop involving $$x, y$$ after we evaluate the linearization at the fixed points. Here is how.
Let's compute the linearization of the right-hand side: $$L(x,y) = \left[ \begin{array}{ll} A - By & -Bx\\ Dy & C - Dx\\ \end{array} \right]$$ Evaluating it at $$(x,y) = (0,0)$$, we get: $$L(0,0) = \left[ \begin{array}{ll} A & 0\\ 0 & C\\ \end{array} \right],$$ so the eigenvalues are $$A, C$$. Since they each have a positive real part (I know, you said they are real and positive, but I am using language suitable for a more general setting), the fixed point $$(x, y) = (0,0)$$ is unstable.
Similarly inspect the eigenvalues of $$L(C/D, A/B)$$.