# “Orientation” of a surface boundary

Given is a surface patch embedded in $R^3$ with disk topology, i.e. it has a single boundary. I would like to know if it is possible to determine if the boundary is "interior" or "exterior" - an intuitive definition is given below, but can it be made mathematically rigorous?

For example, take a sphere that is cut in two by a plane (the two parts need not have equal size). The boundary of the larger part I would classify as "interior", since it could be seen as a hole in the surface (just think of an extreme case in which only a small cap is removed). The boundary of the smaller part would be "exterior". If the sphere is cut exactly along the equator, we get the indeterminate case.

Now, imagine you're walking on the surface along the boundary, so that the surface always lies to your right. Intuitively, the boundary should be classified as "interior" if you make more left turns than right turns on a complete loop, and vice versa.

Is there a way to rigorously define this notion?

EDIT:

I would actually like to have a computational criterion that distinguishes the cases. One idea, which I do not know how to formulate explicitly, would be to integrate along the boundary, summing up the deviations from the straight line in the tangent plane, e.g. a curve to the left counting positively, a curve to the right negatively. The sign of the integral over the entire contour would decide on the "orientation" (sorry I don't have a better term to describe this).

-
What do you mean by "computational criterion", if not what I described? In general, doing integrals is hard -- if you just hand someone an arbitrary smooth function, chances are very, very good that they're not going to be able to symbolically integrate it. As you do more geometry, you'll find that there are a lot of concepts that are very pretty but that can only be explicitly computed for things like spheres and Euclidean space. – Aaron Mazel-Gee May 16 '11 at 15:27
@Aaron: I have, in fact, a very concrete problem: the surface is given as a triangle mesh. I need to find the "holes" in this mesh and somehow distinguish them from the outer boundary – koletenbert May 24 '11 at 7:32
@Aaron: sorry I had only read the first version of your answer... I didn't realize that you completely re-edited it :-) it seems like the new version does what I want, I will give it a try! – koletenbert May 24 '11 at 7:40

One possibility would be to look at the curvature of the boundary curve, "as seen from the surface":

On a smooth curve in $\mathbb{R}^3$, every point has its associated "osculating circle", the circle which closest approximates the curve at that point among all circles in $\mathbb{R}^3$. Associated to this is the curvature vector, which points towards the center of the circle with magnitude $(radius)^{-1}$. (If the curve is locally straight, this osculating circle degenerates to the linear approximation, which is a "circle" with radius $\infty$ and hence curvature $0$.) You can take the component of this curvature vector in the tangent plane (this is where you get the "as seen by the surface" from). This will always be perpendicular to the boundary curve, so it'll be some scalar multiple of the inward-pointing normal curve. The obvious thing to do, then, would be to take the integral of this inward-tangential-curvature function; if it's positive then you're on an exterior curve, and if it's negative then you're on an interior curve. If you think about it, you'll see that this matches up with your intuitive definition on the sphere.

-
You're referring only to the topological/differential structure. It seems that the OP had in mind something that makes use of the metric structure. – joriki May 11 '11 at 22:04
Also, the desired "orientation" in quotation marks in the title is not the orientation you're talking about. Whether you orient the boundary circle clockwise or counterclockwise has nothing to do with the fact that it's an exterior boundary of the disk. – joriki May 11 '11 at 22:09
Oh I see, thank you. I must not have read the question closely enough. I'll edit. – Aaron Mazel-Gee May 11 '11 at 22:30

One possibility that could at least distinguish the two cases you've given as examples, is to demand that an exterior boundary have a kind of convexity: any two points $x$ and $y$ on the boundary should be able to be connected by a minimizing geodesic on the patch which does not intersect the boundary except at $x$ and $y$.

If this is a more restrictive condition for an exterior boundary than you were hoping for, we could try to weaken it by adding a parameter $\epsilon$. Then we could consider a closed curve $\gamma$ which separates the patch $D$ into two pieces $D_1$, and $D_2$, and is such that the boundary is contained in $D_2$, and that the area of $D_2$ is less than $\epsilon$ times the area of $D_1$. We would say that the boundary is exterior if such a $\gamma$ could be found that has the above mentioned convexity property with respect to $D_1$. This idea could be adapted to handle boundaries with multiple components.

-