Analytic solution to coupled nonlinear first order ODEs with quadratic terms on the right hand side

Question

I am trying to find the analytic solution to the following nonlinear, coupled, first order ODE:

$$\frac{dx}{dt} = -2ax^2-bxy$$

$$\frac{dy}{dt} = ax^2-bxy$$

I started with calculating $\frac{dx}{dy}$ and $\frac{dy}{dx}$, but I found $\frac{dx}{dy} \neq (\frac{dy}{dx})^{-1}$, which means we can't change the order of derivative. I failed to decoupled the variables, and I am also not sure if such an analytical solution can be found. I looked at two paper that discussed similar cases:

1. https://doi.org/10.1007/s11040-021-09400-7 This paper discusses a subset of the ODE system with 2nd order polynomial on the right-hand side. Unfortunately, my system is not in that set.
2. https://doi.org/10.1063/5.0011257 This paper is in the citation of the previous one. It only focuses on ODE systems with quadratic term on the RHS, but I haven't fully understood the method yet.

If someone knows how to approach the problem and is willing to give some hints, it will be much appreciated!! Thank you!!

-----------------------------------------------------Update------------------------------------------------------

Inspired by one of my friends, I do the following transformation:

Multiply $\frac{y}{x^2}$ to the first equation and multiply $\frac{1}{x}$ second one, I get:

$$\frac{yx'}{x^2} = -2ay-b\frac{y^2}{x}$$

$$\frac{xy'}{x^2} = ax-by$$

let $u = \frac{y}{x}$,

$$\frac{yx'}{x^2} = -2aux-bu^2x$$

$$\frac{xy'}{x^2} = ax-bux$$

Then, subtract these two equations:

$$\frac{yx'-xy'}{x^2} = \frac{du}{dt} = x(-bu^2-(2a-b)u-a)$$

Similarly, we can also write $$\frac{dx}{dt} = x^2(-2a-u)$$

Then we have

$$\frac{du}{dx} = \frac{1}{x}\frac{-bu^2-(2a-b)u-a}{-2a-u}$$

$$\frac{1}{x} dx= \frac{2a+u}{bu^2+(2a-b)u+a}du$$ $$\int \frac{1}{x} dx = \log{x} = \int \frac{2a+u}{bu^2+(2a-b)u+a}du$$

According to Wolfram, The integration equals:

Let's call it $A(u)$. Then we know $x = e^{A(u)}$. Plug x in the ODE, then:

$$\frac{du}{dt} = e^{A(u)}(-bu^2-(2a-b)u-a)$$

However, analytic form of u can not be calculated by Wolfram... Is there any way to get around it?

-----------------------------------------------------update---------------------------------------------------- The numerical solution for $a=b=1$ is,

If solving the equation analytically is not possible, is there any way to compute the asymptotic behavior directly? (Here, $x$ approaches $0$ and $y$ approaches $0.16303362$)

-----------------------------------------------------Update------------------------------------------------------

I have successfully calculated the equilibrium state, and I do agree with the answer below that there's no analytical form. Thank you for the help guys!

Answer 1

Your system

$$ \begin{cases} \frac{dx}{dt} = -2ax^2-bxy \\ \frac{dy}{dt} = ax^2-bxy \end{cases} $$

can be re-scaled to $$ \begin{cases}\frac{dx}{dt} = -2x^2-cxy \\ \frac{dy}{dt} = x^2-cxy \end{cases}$$ where $c=\frac{b}{a}$ and assuming $a \neq 0$. Then we have $$\frac{dy}{dx}=\frac{x^2-cxy}{-2x^2-cxy}=\frac{1-c \frac{y}{x}}{-2-c \frac{y}{x}}$$ this is an Homogeneous differential equation and can be solved with $v=\frac{y}{x} \Rightarrow y'=v+xv'$: $$v+x\frac{dv}{dx}=\frac{1-cv}{-2-cv} \Rightarrow \frac{dv}{dv} =\frac{1}{x} \frac{(c-2)v-1-cv^2}{2+cv}$$ from where we get $$ \ln x + C = \int \frac{2+cv}{(c-2)v-1-cv^2}dv$$ and for general $c$ we should analyse different possibilities for $\sqrt{c^2-8c+4}$. Take for example $c=8$, then we have the solution $$\frac{(4y-x)^2}{(2y-x)^3}=C$$ If in the original system we take $z=4y-x$ the new system becomes $$ \begin{cases}y'=z^2-16y^2 \\ z'=6z^2-24zy \end{cases}$$ and using $$\frac{(4y-x)^2}{(2y-x)^3}=C \Rightarrow \frac{z^2}{(2y-x)^3}=C \Rightarrow y=\frac{z-C^{-\frac{1}{3}}z^{\frac{2}{3}}}{2}$$ we obtain for $z$: $$z'=12C^{-\frac{1}{3}}z^{\frac{5}{3}}-6z^2$$ this is an Separable ODE and the solution can be found with $t$ as a function of $z$. Unfortunately this function can't be inverted to write $z$ as a function of $t$. Even if we could do so (may be for an different value of $c$), the equation for $y$ is an standard Riccati equation of the form $$y'=y^2+f(x)$$ and exact solutions can only be found for some specifics $f(x)$.

---