Poincaré-Bendixon show periodic solutions. 
Show that the system $x^{'}=x-y-x^{3}$,$y^{'}=x+y-y^{3}$ has a periodic solution.

I converted to polar:
$r r^{'}=x^{'}x+y^{'}y.$
Thus
$r r^{'}=x^{2}-x^{4}+y^{2}-y^{4}.$
Collecting squares:
$rr^{'}=x^{2}+y^{2}-(x^{2}+y^{2})^{2} +2x^{2}y^{2}.$
Substitute:
$r^{'}=r-r^{3}(1-(\sin^{2}(2\theta))/2)$
$r^{'}=r-r^{3}(1+\cos^{2}(2\theta))/2.$
Setting $r^{'}=0$ yields
$r^{2}=2/(1+\cos^{2}(2\theta).$
Thus $r_{max}= 2^{(1/2)}$ and $r_{min}=1.$
I suck at trig and this doesn't look right. Clearly $(0,0)$ is the only fixed point, so I have a bounded region. I just don't think I have found correct values for $r_{min}$ and $r_{max}$ to find my limit cycle.
 A: What you've done so far is correct, but your conclusions are not quite right. You found that
$$ \dot{r} = r - \frac{r^3}{2} ( 1 + \cos^2(2 \theta) ) $$
When you found that " $r_{min} = 1$ ", what you should do is show $\dot{r} > 0$ when $r < 1$. When you found that " $r_{max} = \sqrt{2}$ ", what you should truly do is show that $\dot{r} < 0$ when $r > \sqrt{2}$.
Then, if you have a system whose initial radius lies in the interval $(1,\sqrt{2})$, then the radius of any point on its trajectory must lie in the interval $[1, \sqrt{2}]$. Thus the trajectory is "trapped" in this region. Now we can apply the Poincare-Bendixson theorem. The point here is that even when $\dot{r} = 0$, this doesn't mean we have a fixed point, unless the angle $\theta$ is also unchanging. Indeed, you noted that the origin is the only fixed point, so by the Poincare-Bendixson theorem, we have a bounded trajectory with no fixed points; there must be a limit cycle. 
A: Dynamical system
Find the periodic solution for the dynamical system:
$$
%
\begin{align}
%
 \dot{x} &= x - y - x^{3} \\
%
 \dot{y} &= x + y - y^{3} \\
%
\end{align}
\tag{1}
%
$$
Fixed points
There is a lone fixed point at the origin
$$
\left[ \begin{array}{c}
 \dot{x} \\ \dot{y}
\end{array} \right]_{(0,0)}
=
\left[ \begin{array}{c}
 0 \\ 0
\end{array} \right]
$$
Methods
Compute $\dot{r}$.
Use the polar coordinate transform
$$
%
\begin{align}
%
 x &= r \cos \theta \\
%
 y &= r \sin \theta \\
%
\end{align}
%
$$
which implies
$$
  r^{2} = x^{2} + y^{2}
\tag{2}
$$
Compute the derivative with respect to time for $(2)$ and use the definitions in $(1)$. This leads to
$$
 \dot{r} = r - r^{3} \left( \cos^{4} \theta + \sin^{4} \theta \right) 
= r 
\left( 
 1 - \frac{1}{4} \left( 3 + \cos 4 \theta \right) r^{2} 
\right)
\tag{3}
$$
Trapping region
To categorize the flow, identify regions where $r$ is expanding or shrinking. Consider the limiting values 
$$1 \le \cos 4\theta \le 1.$$


*

*$\cos 4\theta = 1$
Equation $(3)$ reduces to 
$$
 \dot{r} 
= r \left( 
  1 -  r^{2}
 \right)
$$
The conclusion is
$$
\begin{cases}
  \color{blue}{\dot{r} > 0} &  \color{blue}{r < 1} \\ 
  \color{red}{\dot{r} < 0} &  \color{red}{r > 1} \\ 
\end{cases}
$$


*$\cos 4\theta = -1$
Equation $(3)$ reduces to 
$$
 \dot{r} 
= r \left( 
  1 -  \frac{1}{2}r^{2}
 \right)
$$
This implies
$$
\begin{cases}
  \color{blue}{\dot{r} > 0} &  \color{blue}{r < \sqrt{2}} \\ 
  \color{red}{\dot{r} < 0} &  \color{red}{r > \sqrt{2}} \\ 
\end{cases}
$$
The figures below depict the zone structure for the two limiting conditions. Combine these zone structures and identify where, in both figures, is $\dot{r}$ always positive, and always negative. There will be an annulus where the sign of $\dot{r}$ isn't specified. The third figure shows this trapping region in gray. 

The trapping region is the set $ 1 \le r \le \sqrt{2}$
Results
The flow field is plotted next. The dashed, red lines are the nullclines. The gray annulus is the trapping region.

