# Proof that every polygon with an inscribed circle is convex?

In many elementary (and not-so-elementary) Euclidean geometry texts, a (simple) polygon is said to be tangential  if it is convex and has an inscribed circle (i.e., a circle that intersects and is tangent to each side of the polygon). The assumption of convexity is not needed: I've come up with a rather laborious proof that every polygon with an inscribed circle is convex. But I'd like to find either a simple elementary proof or a reference to a proof in the literature. (By "elementary," I mean using only standard facts of axiomatic Euclidean geometry.)

Does anyone know of a reference for a proof of this fact (elementary or not)? Or can anyone think of a straightforward elementary proof? You can use any definition of "convex polygon" that you like, but the easiest one to work with is that for each edge, the vertices not on that edge lie on one side of the line through that edge.

(Interestingly, the corresponding fact for circumscribed circles--i.e., that every polygon with a circumscribed circle is convex--is quite easy to prove: If P has a circumscribed circle, any two nonadjacent sides of P are non-intersecting chords of the circle; it is easy to show that both endpoints of each chord lie on the same side of the line through the other, and from there it is an easy matter to prove that P is convex.)

-
Isn't it enough to prove that the angle at each vertex is less than 180 degrees, and isn't that obvious? –  Qiaochu Yuan Sep 17 '10 at 21:26
You must be restricting yourself to plane simple polygons otherwise it seems to me that the pentagram (regular but self-intersecting) admits a circle which is tangent to all 5 sides of the polygon. –  Joseph Malkevitch Sep 17 '10 at 21:49
Qiaochu: To use "angles less than 180 degrees" as a criterion for convexity, you have to distinguish interior vs. exterior angles at each vertex. It's obvious that one of the angles at each vertex is less than 180 degrees, but if the polygon is not known a priori to be convex, it's not obvious (at least to me) that it's the interior angle that's less than 180. –  Jack Lee Sep 17 '10 at 22:43
Joseph: Yes, I'm only interested in simple polygons. I added that to the question. –  Jack Lee Sep 17 '10 at 22:45

The definition of simple establishes that the polygon itself equals the intersection of the half-planes tangent to the circle at the points where the polygon's sides contact the circle. Any nonempty intersection of halfplanes is convex.

-
I'm not convinced: Given an edge AB, how do you show that each other nonadjacent edge is contained in the half-plane determined by AB & the circle? –  Jack Lee Sep 18 '10 at 4:36
@Jack Lee: it's enough to see that all vertices lie within the intersection of half-planes. If there is some external vertex, it is separated by some line from the points where its defining edges meet the circle. (Notice that the intersection of half-planes contains the circle.) But this implies those defining edges are not adjacent, whence they could not form the vertex in the first place. –  whuber Sep 20 '10 at 15:08
@whuber: I don't think this argument is quite sufficient. Look at this diagram: math.washington.edu/~lee/inscribed.png . The vertex A lies on the "wrong side" of the line DE, and is separated by that line from the two points where its defining edges meet the circle. But the defining edges are still adjacent. The moral is that it's not sufficient just to look at a couple of vertices and their adjacent edges; you have to somehow consider the entire polygon. –  Jack Lee Sep 24 '10 at 18:05
@Jack Lee: Right, but the point A is never under consideration; it cannot be a vertex of the polygon if line DE contains an edge. Of course if you want a full proof with all details given (an attitude that I respect, because it has frequently led to new insights and new mathematics) you won't find it in my response, which was structured to indicate the essence of one approach. But I don't think it overlooks any difficult or essential detail. –  whuber Sep 24 '10 at 18:13
@whuber: You wrote "the point A is never under consideration; it cannot be a vertex of the polygon if line DE contains an edge." I believe this is true, but how do you prove it? This is exactly the crux of the issue, and I think it's more subtle than most people realize. –  Jack Lee Sep 24 '10 at 22:53

edit 2: Suppose that a simple polygon has an inscribed circle. Without loss of generality, pick a "first" edge and let the circle be on the "right" side of that edge. The angle that is on the same side as the circle--that is, to the right--between the first edge and the second edge must have measure less than 180°. Similarly, the angle between the second and third edges that is also on the right must also have measure less than 180°, and so on, so that all of the angles on the right side of the polygon's perimeter (whether it is the inside or the outside) must have measure less than 180° and all of the angles on the left side must have measure greater than 180°.

Since the sum of the interior angles of a simple polygon with n sides is 180°(n – 2), the average measure of an interior angle of a simple polygon with n sides is 180° – 360°/n, which is strictly less than 180°. So, since the angles on the left side of the perimeter all have measure greater than 180°, their average is greater than 180°, so the left cannot be the interior of the polygon and the right side must be the interior, so the inscribed circle must be in the interior of the polygon and the internal angles all have measure less than 180°.

Assuming the polygon is non-self-intersecting, then two consecutive sides of the polygon correspond to two consecutive points on tangency on the circle, with the angle formed by the two sides subtending the minor arc between the points of tangency. The measure of the angle of the polygon is half the difference between the measure of the major and minor arcs between the points of tangency. The greatest possible difference would be the degenerate case where the minor arc has measure 0° and the major arc has measure 360°, giving the angle measure 180°; for non-degenerate cases, the difference in the arc measures must be less than 360°, so the angle measure must be less than 180°. This applies to every pair of consecutive sides, so every interior angle of the polygon has measure less than 180°, so the polygon is convex.

edit: Alternately, and blatantly assuming that the inscribed circle must be in the interior of the polygon, suppose that a polygon is not convex, so that there is a vertex at which the interior angle has measure greater than 180°. For a circle to be tangent to the two sides that meet at that vertex, the circle must be exterior to the polygon at that vertex (so as to be in the non-reflex side of that angle) and thus cannot be the inscribed circle.

-
Isaac: This proves that one of the angles at each vertex has measure less than 180 degrees; but I don't see how it proves that it's the interior angle. If you don't know a priori that the polygon's convex, it's very tricky to decide which angle at a given vertex is the interior angle and which is the exterior angle -- in principle, you have to know the entire polygon to decide. –  Jack Lee Sep 17 '10 at 22:48
@Jack: Doesn't the inscribed circle have to be in the interior of the polygon? I think that would be sufficient to say that the angles I proved were of measure less than 180° are the interior angles. –  Isaac Sep 17 '10 at 23:44
Isaac: Yes, it's true that the inscribed circle has to be in the interior, but how do you prove it? This is the crux of the issue, I think. It can be subtle -- for instance, the link below shows an example of a non-convex simple polygon such that all of the lines through the edges are tangent to a circle, and such that the circle is not in the interior of the polygon. Somehow, you have to use the fact that each edge actually makes contact with the circle. Here's the link: math.washington.edu/~lee/hat.gif –  Jack Lee Sep 18 '10 at 0:02
@Jack: How about my argument now (edit 2)? –  Isaac Sep 18 '10 at 1:32
Isaac: This looks more promising. I'm still a little unclear about how to make rigorous sense, in elementary terms, of what it means to be "on the right side" (or more to the point, "on the same side") of the perimeter. (And, for that matter, I still don't have a good proof that "all interior angles less than 180 degrees" implies any of the other characterizations of convexity; but that's a different problem.) –  Jack Lee Sep 18 '10 at 5:13

Thanks to everyone who suggested approaches to this problem. In the end, none of the suggested approaches fit into the axiomatic framework that I was working in, so I had to write up my own rather laborious proof. It's a bit long to post here, but in order to close this question, I just want to post the reference and a quick summary of the approach.

You can find the complete proof in my textbook Axiomatic Geometry (Theorem 14.31). The basic idea is first to prove the following lemma:

Lemma. Let $\mathscr P$ be a polygon circumscribed about a circle $\mathscr C$. Suppose $A$ is any vertex of $\mathscr P$, and $E$ and $F$ are the points of tangency of the two edges containing $A$. Then there are no points of $\mathscr P$ in the interior of $\triangle AEF$.

To prove that a tangential polygon $\mathscr P$ must be convex, the basic idea is to show that if $\ell$ is any edge line of $\mathscr P$, then there can't be any vertices of $\mathscr P$ on the "wrong" side of $\ell$ (the side not containing the inscribed circle), because that would violate the lemma.

-