Periodic Orbit on a Nonlinear System Let the following ODE system be
\begin{align}
x'&=x-y+x^2-2x(x^2+y^2)^\frac{1}{2}
\\
y'&=x+y+xy-2y(x^2+y^2)^{\frac{1}{2}}
\end{align}
We can prove easily that $(0,0)$ is the unique equilibrium point. Using the polar coordinates, we can rewrite the system as follows:
\begin{align}
\rho'&=\rho+\rho^2(-2+\cos\theta)
\\
\theta'&=1
\end{align}
Working with the system (for $\theta$ is simple, for $\rho$ we must proceed as a Bernoulli equation), we arrange the following solutions:
\begin{align}
\rho(t)&=\frac{1}{\frac{\cos(\theta_{0}+t)+\sin(\theta_{0}+t)}{2}+2+Ce^{-t}}
\\
\theta&=t+\theta_{0}
\end{align}
Where we can define C for $t=0$ as $C=\frac{1}{\rho_{0}}-2-\frac{\cos\theta_{0}+\sin\theta_{0}}{2}$
The question is, once we have arrived here and we convert into Cartesian coordinates using $\theta_{0}=\arctan(\frac{y_{0}}{x_{0}})$; $\rho_{0}=(x_{0}^2+y_{0}^2)^\frac{1}{2}$, and $x=\rho\cos\theta$; $y=\rho\sin\theta$, how can I find a periodic solution in that system?
 A: Starting from the Pablo Luis's result (I didn't check it) : 
\begin{align}
\rho(t)&=\frac{1}{\frac{\cos(\theta_{0}+t)+\sin(\theta_{0}+t)}{2}+2+Ce^{-t}}
\\
\theta&=t+\theta_{0}
\end{align}
Obviously the solution is not periodic due to the term $Ce^{-t}$.
But for large $t$ , that is a long time after the start, $Ce^{-t}\to 0$. The solution tends to a periodic function :
$$\rho(t)\simeq\frac{1}{\frac{\cos(\theta_{0}+t)+\sin(\theta_{0}+t)}{2}+2}$$
This is proved by 
$$\rho(t+2k\pi)\simeq\frac{1}{\frac{\cos(\theta_{0}+t+2k\pi)+\sin(\theta_{0}+t+2k\pi)}{2}+2}=\frac{1}{\frac{\cos(\theta_{0}+t)+\sin(\theta_{0}+t)}{2}+2}$$
$$\rho(t+2k\pi)\simeq\rho(t)\qquad t\:\text{ large.}$$
On Cartesian system :
$$\begin{cases}
x(t)\simeq \frac{\cos(\theta_{0}+t)}{\frac{\cos(\theta_{0}+t)+\sin(\theta_{0}+t)}{2}+2}\simeq x(t+2k\pi) \\
y(t)\simeq \frac{\sin(\theta_{0}+t)}{\frac{\cos(\theta_{0}+t)+\sin(\theta_{0}+t)}{2}+2}\simeq y(t+2k\pi) 
\end{cases}\qquad t\:\text{ large.}$$
Eliminating $t$ from the above parametric solution leads to the equation of the limit trajectory which is an ellipse :
$$15(x^2+y^2)-2xy+4(x+y)=4$$

NOTE : 
LutzL points out a sign mistake in the Pablo Luis's equation from which I started my answer (first above equation). Even if it was not correct, the method shows how to proceed in order to find the equation of the steady-state trajectory.
Comparing to the graph from LutzL, apparently one have to rotate the axes of $\pi$ to be consistent with the graph in my answer. The semi-axis lengths $a$ and $b$ agree. The center is symmetrical with respect to the origin. 
A: Obviously, the period has to be $2\pi$. You should easily see that $ρ(2\pi)=ρ(0)$ demands that $C=0$. That then is all.

For the radius equation I get
$$
ρρ'=xx'+yy'=ρ^2(1+x)-2ρ^3 \implies ρ'=ρ-ρ^2(2-\cos θ)
$$
there is no additional sine term. Everything else apart from this change remains the same.

red, blue: numerical solutions, gray: $ρ(θ)=\frac2{4-\cosθ-\sinθ}$
