Can a hyperbolic fixed point be inside a homoclinic loop on a plane?

Consider a system of odes on a plane. It is obvious that a fixed point of centre-type stability (imaginary e-values) can exist inside a homoclinic loop to a hyperbolic saddle fixed point. For example, take the Hamiltonian $H(x,y) = \frac{1}{2}(y^{2}-x^{2}) + x^{3}$. Here the hyperbolic saddle point at origin has a "fish" shaped homoclinic loop enclosing the centre fixed point at $(1/3, 0)$.

What I'm wondering is whether a homoclinic loop can enclose fixed points other than centre type, i.e. can a "sink / source" or a "saddle" exist inside a homoclinic loop? And can this situation be generalised to $\mathbb{R}^{n}$?

I cannot think of a way to either prove / disprove this. I suspect Poincare-Bendixson and index theory could be used, but not sure how.

• For a sink/source example see this link. A single saddle can't be inside the homoclinic loop on plane: this follows from index theory or Poincare-Bendixson theory. From index theory follows that inside a homoclinic loop you have to have, for example, one saddle and two sinks. This can be done using the same idea from my answer: start with suitable conservative system, perturb it everywhere except at chosen homoclinic loop and you are done. Feb 22 '18 at 15:38
• Evgeny, thank you. However, one thing is unclear. You say "from index theory follows that inside a homoclinic loop you have to have one saddle and two sinks". Why is that? If one considers a homoclinic orbit as a periodic orbit, then indeed this result will follow, as the index of a periodic orbit is $1$, the index of a sink is $1$ and the index of a saddle is $-1$. However, it is pointed out in literature that "a homoclinic orbit should $\mathbf{not}$ be treated as a periodic orbit for index theory" (see Wiggins, "Introduction to applied nonlinear dynamical system and chaos"). @Evgeny
– Alex
Feb 22 '18 at 17:13
• Yeah, to some extent you are right. I looked at Wiggins' note and the example that he is referring. It relies heavily on the non-hyperbolicity of an equilibrium. When I was answering, I was thinking about ordinary hyperbolic saddle and I have the feeling that in this case there is still a workaround. Let me take a look at another reference. Feb 22 '18 at 19:05
• Well, we can't apply index theory straightforwardly to homoclinic loop: vector field vanishes at equilibrium and the homoclinic loop itself is usually only piecewise smooth instead of smooth. I thought that it was possible to "smooth out" homoclinic loop and obtain nice smooth curve with non-vanishing vector field and the index of that curve would be $+1$. That was my line of reasoning for hyperbolic saddle. What I currently don't understand is why do I get the same $+1$ index for an equilibrium at figure 3-18 here. Feb 22 '18 at 20:05
• Yes, I also thought that lack of global smoothness of the homoclinic loop and the presence of an equilibrium point is an obstruction to index theory @Evgeny
– Alex
Feb 22 '18 at 20:17

For a planar flow generated by a vector field $\vec v$, with a homoclinic loop $\ell$ based at a saddle point $p$, and letting $D$ be the 2-disc bounded by $\ell$, one can apply index theory indirectly to conclude that the index sum over the singularities of $\vec v$ in the interior of $D$ must equal $+1$. And then one can construct examples where there is a saddle singularity on the inside, although of course that saddle singularity must be balanced appropriately with other singularities, including at least one of positive index, so that the index sum equals $+1$. For example, one saddle and two center singularities can be obtained (I'll say more about this below).
To carry out the index theory argument, one chooses a small enough neighborhood $U$ of $p$ which is disjoint from all the zeroes of $\vec v$ except $p$ itself, and one then simultaneously perturbs $D$ and $\vec v \mid D$ near $U$ to obtain a new disc $D'$ and new vector field $\vec v'$ on $D'$, so that the boundary of $D'$ is smooth, $\vec v'$ is tangent to the boundary of $D'$, $\vec v'$ has no zeroes in $U \cap D'$, and outside of $U$ we have $D-U=D'-U$, $\vec v \mid D-U = \vec v' \mid D'-U$. Thus the sum of the indices of the zeroes of $\vec v$ in $D$ equals the sum of the indices of the zeroes of $\vec v'$ in $D'$ which equals $+1$.
And then you can work backwards. Start, for example, from the example given in the link of the first comment of @Evgeny, which is defined on a nice smooth round disc $D'$ with 2 centers and 1 saddle (index sum $+1$). In this example the vector field $\vec v$ points inward along the boundary of $D'$. So, first one will have to alter that example so that the vector fields points tangentially along the boundary of $D'$; one can, for example rotate the vector field along and near the oundary of $D'$ to achieve this purpose. After that, one can do a further alteration, "unperturbing" the example near a point on the boundary, to create a saddle singularity and homoclinic loop.