# Proof that every polygon with an inscribed circle is convex?

In many elementary (and not-so-elementary) Euclidean geometry texts, a (simple) polygon is said to be tangential  if it is convex and has an inscribed circle (i.e., a circle that intersects and is tangent to each side of the polygon). The assumption of convexity is not needed: I've come up with a rather laborious proof that every polygon with an inscribed circle is convex. But I'd like to find either a simple elementary proof or a reference to a proof in the literature. (By "elementary," I mean using only standard facts of axiomatic Euclidean geometry.)

Does anyone know of a reference for a proof of this fact (elementary or not)? Or can anyone think of a straightforward elementary proof? You can use any definition of "convex polygon" that you like, but the easiest one to work with is that for each edge, the vertices not on that edge lie on one side of the line through that edge.

(Interestingly, the corresponding fact for circumscribed circles--i.e., that every polygon with a circumscribed circle is convex--is quite easy to prove: If P has a circumscribed circle, any two nonadjacent sides of P are non-intersecting chords of the circle; it is easy to show that both endpoints of each chord lie on the same side of the line through the other, and from there it is an easy matter to prove that P is convex.)

• Isn't it enough to prove that the angle at each vertex is less than 180 degrees, and isn't that obvious? – Qiaochu Yuan Sep 17 '10 at 21:26
• You must be restricting yourself to plane simple polygons otherwise it seems to me that the pentagram (regular but self-intersecting) admits a circle which is tangent to all 5 sides of the polygon. – Joseph Malkevitch Sep 17 '10 at 21:49
• Qiaochu: To use "angles less than 180 degrees" as a criterion for convexity, you have to distinguish interior vs. exterior angles at each vertex. It's obvious that one of the angles at each vertex is less than 180 degrees, but if the polygon is not known a priori to be convex, it's not obvious (at least to me) that it's the interior angle that's less than 180. – Jack Lee Sep 17 '10 at 22:43
• Joseph: Yes, I'm only interested in simple polygons. I added that to the question. – Jack Lee Sep 17 '10 at 22:45

Thanks to everyone who suggested approaches to this problem. In the end, none of the suggested approaches fit into the axiomatic framework that I was working in, so I had to write up my own rather laborious proof. It's a bit long to post here, but in order to close this question, I just want to post the reference and a quick summary of the approach.

You can find the complete proof in my textbook Axiomatic Geometry (Theorem 14.31). The basic idea is first to prove the following lemma:

Lemma. Let $\mathscr P$ be a polygon circumscribed about a circle $\mathscr C$. Suppose $A$ is any vertex of $\mathscr P$, and $E$ and $F$ are the points of tangency of the two edges containing $A$. Then there are no points of $\mathscr P$ in the interior of $\triangle AEF$.

To prove that a tangential polygon $\mathscr P$ must be convex, the basic idea is to show that if $\ell$ is any edge line of $\mathscr P$, then there can't be any vertices of $\mathscr P$ on the "wrong" side of $\ell$ (the side not containing the inscribed circle), because that would violate the lemma.

The definition of simple establishes that the polygon itself equals the intersection of the half-planes tangent to the circle at the points where the polygon's sides contact the circle. Any nonempty intersection of halfplanes is convex.

• I'm not convinced: Given an edge AB, how do you show that each other nonadjacent edge is contained in the half-plane determined by AB & the circle? – Jack Lee Sep 18 '10 at 4:36
• @Jack Lee: it's enough to see that all vertices lie within the intersection of half-planes. If there is some external vertex, it is separated by some line from the points where its defining edges meet the circle. (Notice that the intersection of half-planes contains the circle.) But this implies those defining edges are not adjacent, whence they could not form the vertex in the first place. – whuber Sep 20 '10 at 15:08
• @whuber: I don't think this argument is quite sufficient. Look at this diagram: math.washington.edu/~lee/inscribed.png . The vertex A lies on the "wrong side" of the line DE, and is separated by that line from the two points where its defining edges meet the circle. But the defining edges are still adjacent. The moral is that it's not sufficient just to look at a couple of vertices and their adjacent edges; you have to somehow consider the entire polygon. – Jack Lee Sep 24 '10 at 18:05
• @Jack Lee: Right, but the point A is never under consideration; it cannot be a vertex of the polygon if line DE contains an edge. Of course if you want a full proof with all details given (an attitude that I respect, because it has frequently led to new insights and new mathematics) you won't find it in my response, which was structured to indicate the essence of one approach. But I don't think it overlooks any difficult or essential detail. – whuber Sep 24 '10 at 18:13
• @whuber: You wrote "the point A is never under consideration; it cannot be a vertex of the polygon if line DE contains an edge." I believe this is true, but how do you prove it? This is exactly the crux of the issue, and I think it's more subtle than most people realize. – Jack Lee Sep 24 '10 at 22:53

edit 2: Suppose that a simple polygon has an inscribed circle. Without loss of generality, pick a "first" edge and let the circle be on the "right" side of that edge. The angle that is on the same side as the circle--that is, to the right--between the first edge and the second edge must have measure less than 180°. Similarly, the angle between the second and third edges that is also on the right must also have measure less than 180°, and so on, so that all of the angles on the right side of the polygon's perimeter (whether it is the inside or the outside) must have measure less than 180° and all of the angles on the left side must have measure greater than 180°.

Since the sum of the interior angles of a simple polygon with n sides is 180°(n – 2), the average measure of an interior angle of a simple polygon with n sides is 180° – 360°/n, which is strictly less than 180°. So, since the angles on the left side of the perimeter all have measure greater than 180°, their average is greater than 180°, so the left cannot be the interior of the polygon and the right side must be the interior, so the inscribed circle must be in the interior of the polygon and the internal angles all have measure less than 180°.

Assuming the polygon is non-self-intersecting, then two consecutive sides of the polygon correspond to two consecutive points on tangency on the circle, with the angle formed by the two sides subtending the minor arc between the points of tangency. The measure of the angle of the polygon is half the difference between the measure of the major and minor arcs between the points of tangency. The greatest possible difference would be the degenerate case where the minor arc has measure 0° and the major arc has measure 360°, giving the angle measure 180°; for non-degenerate cases, the difference in the arc measures must be less than 360°, so the angle measure must be less than 180°. This applies to every pair of consecutive sides, so every interior angle of the polygon has measure less than 180°, so the polygon is convex.

edit: Alternately, and blatantly assuming that the inscribed circle must be in the interior of the polygon, suppose that a polygon is not convex, so that there is a vertex at which the interior angle has measure greater than 180°. For a circle to be tangent to the two sides that meet at that vertex, the circle must be exterior to the polygon at that vertex (so as to be in the non-reflex side of that angle) and thus cannot be the inscribed circle.

• Isaac: This proves that one of the angles at each vertex has measure less than 180 degrees; but I don't see how it proves that it's the interior angle. If you don't know a priori that the polygon's convex, it's very tricky to decide which angle at a given vertex is the interior angle and which is the exterior angle -- in principle, you have to know the entire polygon to decide. – Jack Lee Sep 17 '10 at 22:48
• @Jack: Doesn't the inscribed circle have to be in the interior of the polygon? I think that would be sufficient to say that the angles I proved were of measure less than 180° are the interior angles. – Isaac Sep 17 '10 at 23:44
• Isaac: Yes, it's true that the inscribed circle has to be in the interior, but how do you prove it? This is the crux of the issue, I think. It can be subtle -- for instance, the link below shows an example of a non-convex simple polygon such that all of the lines through the edges are tangent to a circle, and such that the circle is not in the interior of the polygon. Somehow, you have to use the fact that each edge actually makes contact with the circle. Here's the link: math.washington.edu/~lee/hat.gif – Jack Lee Sep 18 '10 at 0:02
• @Jack: How about my argument now (edit 2)? – Isaac Sep 18 '10 at 1:32
• Isaac: This looks more promising. I'm still a little unclear about how to make rigorous sense, in elementary terms, of what it means to be "on the right side" (or more to the point, "on the same side") of the perimeter. (And, for that matter, I still don't have a good proof that "all interior angles less than 180 degrees" implies any of the other characterizations of convexity; but that's a different problem.) – Jack Lee Sep 18 '10 at 5:13

Below is a rather more complete version of the sketch in my previous answer. However, since I do not have access to your book, I don't know how it compares to yours in length.

Definition. Let $P_n$ denote a planar $n$-gon with $n\ge3$ an integer. Let its vertices be $V_m$, where $m$ is an integer satisfying $1\le m\le n$. We say that:

1. $P_n$ is is cyclic provided all its vertices (and only its vertices) intersect a unique circle.

2. $P_n$ is tangential iff each one of its sides intersects a unique circle with centre $C$ at the points $N_m$, so that $C\widehat N_mV_m=90°$.

Lemma.

1. If two points $P$ and $Q$ are on a circle $\partial D$ centred at $C$, then the segment $PQ$ is in the region $\partial D\cup\operatorname{int}\partial D=D$, the closed disc bounded by the circle.

2. If a segment $PQ$ intersects a circle with centre $C$ at a point $N$ such that $C\widehat NP$ is a right angle, then all points of $PQ\ne N$ lie outside $D$, namely the exterior of $\partial D$.

Proof.

1. If $P,Q$ and $C$ are collinear, then $PQ$ is a diameter of $\partial D$, then $PQ\subset D$ since $|PC|=|CQ|$ is the radius of $\partial D$. Now suppose that the points $P,C,Q$ are non-collinear. Then we have that $\triangle PCQ$ is isosceles. Thus, the farthest distance from $C$ to $PQ$ is along $CP$ or $CQ$, radii of $\partial D$. Therefore for any point $X\in PQ$, we have $|CX|\le|CP|=|CQ|$, so that all such $X$ lie in $D$.

2. Since $C\widehat NP=90°$, we have that $|CN|\le|CX|\,\,\forall \,\,X\in PQ$. Since $|CN|$ is the radius of $\partial D$, it follows that all points $X\ne N$ are in the exterior of $\partial D$.

Theorem. A polygon is convex whenever it is tangential.

Proof. All the sides of $P_n$ are in $\operatorname{ext}\partial D \cup \partial D$ (by the second lemma) and connected at the vertices $V_m$. These vertices are unique and contain only one internal angle of $P_n$ (since the tangent line through a point on a circle is unique, and two nonparallel lines intersect in at most one point). Now consider the simple cyclic $n$-gon $R_n$ whose vertices are $N_m$, which we call the cyclic dual of $P_n$ (that $R_n$ exists is clear; it is unique since it must have the points $N_m$ as vertices and no two nonconsecutive sides must intersect). Then each side $N_mN_{m+1}$ of $R_n$ is in $D$ (by the first lemma), so that $R_n\subset \operatorname{int}P_n\cup P_n$. Thus $P_n$ and $R_n$ intersect only at the points $N_m$. Now consider the vertex $V_m$ of $P_n$. Since $V_m$ is between $N_m$ and $N_{m+1}$, and $N_m,V_m$ and $N_{m+1}$ are non-collinear (one is outside $\partial D$ and the others on it), we have the $\triangle N_mV_mN_{m+1}$, so that $$N_m\widehat V_mN_{m+1}<180°\forall m. \square$$

If you agree with the definition given (and I don't see how not since any definition of tangency would have to specify that if a circle and a segment are tangent then the radius of the circle is perpendicular to the segment, no matter how disguised), then you must also agree that this proof is valid, unless of course it demonstrably is not.

Two notes on the OP:

1. You observe that the assumption of convexity is not needed in the definition of tangential polygon; indeed, the assumption of simplicity is not needed too, for every tangential polygon must perforce be simple (I have no elementary proof of this, but a heuristic to see why this is the case is that tangency of a smooth connected curve by a polygonal path is a strong condition, whose essence is in approximating the curve as closely as possible; thus the polygon must have all the properties of the curve independent of its smoothness -- like convexity or simplicity, in this case).

2. You remark that the corresponding theorem for cyclic polygons is simpler to prove, and give a suggestion as to how one might see this. While I agree with the observation, your attempt at a proof sketch is invalid, for one of the statements is false. Indeed, you say, 'If P has a circumscribed circle, any two nonadjacent sides of P are non-intersecting chords of the circle...' But this is false since cyclicity of P is not strong enough to guarantee simplicity, which you tacitly assume in that statement (there exists cyclic, non-simple polygons). The only thing that follows from cyclicity is convexity (and compactness), so that approximating a circle (or in general, any smooth, simple, convex, compact planar curve) from its interior is much more messy and not guaranteed to work than approximating from without.

• I don't understand the reasoning in your last paragraph. It seems to say that for every vertex of the polygon, the meeting point of the normals to the lines adjacent to that vertex will lie on the side of that angle corresponding to the interior of the polygon. But the argument in the preceding paragraph showed exactly that if one of the angles is nonconvex, then the normals to the sides adjacent to that angle will NOT meet on the side corresponding to the interior of the polygon. That's exactly what happens at vertex C in this picture. – Jack Lee May 31 '18 at 22:47
• @JackLee The last paragraph simply states that for any vertex of the nonconvex polygon, every pair of normals each to a respective pair of adjacent sides satisfies one and only one of the two conditions: either they meet inside the region bounded by the polygon or they meet outside that region. From this it follows that there cannot be a set of $n$ normals, one per side, so that they all meet at one point only. – Allawonder May 31 '18 at 22:56
• @JackLee Using your picture, what I'm saying for example is that any normals to $AD$ and $AB$ must meet in the region bounded by the quadrilateral $ABCD$ and any normals to $CD$ and $CB$ always meet outside the polygonal region, so that the four normals never concur. – Allawonder May 31 '18 at 23:01
• I don't see how that follows -- it's perfectly possible for normals to AD and AB in my picture to meet outside the polygonal region, if the points of tangency are sufficiently close to D and B. The problem is that a point of intersection being on the reflex or nonreflex side of an angle doesn't necessarily correspond to being in the interior or exterior of the polygonal region. – Jack Lee May 31 '18 at 23:04
• @JackLee Oh, that seems to be true. It's quite late here. I shall look at it again tomorrow. – Allawonder May 31 '18 at 23:17

Here is the sketch of a much shorter proof.

If a (simple) polygon is tangential, then it is convex (all its angles are each less than two right angles). Let the polygon be $n$-sided. Then draw radii from the centre of the circle to the $n$ points of tangency (by hypothesis). This splits the $n$-gon into $n$ quadrilaterals which are kites (that they are kites follows from the $RHS$ congruency test, after having connected each vertex of the $n$-gon to the centre of the circle). Each of these kites has a vertex of the $n$-gon as a vertex and (by a well-known result of Euclidean Geometry about tangent straight lines to circles and radii thereof) two right angles. The last angle of each kite is formed at the centre of the circle by two radii. Since the sum of angles in a quadrilateral is four right angles, it is clear that the two non-right angles in each quadrilateral must be supplementary, so that either one of them is always less than two right angles. The result follows.

• The trouble with this argument is that unless you know in advance that the polygon is convex, it's not easy to verify that the interior angles of the polygon coincide with the interior angles of the quadrilaterals. It's true that the interior angles of the quadrilaterals measure less than $180^\circ$; but the difficult part is ruling out the possibility that an interior angle of the original polygon coincides with an exterior angle of one of these quadrilaterals. – Jack Lee May 31 '18 at 19:28
• @JackLee I don't quite understand what you say the problem is. How could the angles of the original polygon possibly coincide with anything apart from the largest angle of the kites, following my description? – Allawonder May 31 '18 at 20:58
• @JackLee Perhaps you're not keeping the fact of tangency in mind. I'll give an even shorter proof using this explicitly as a separate answer. – Allawonder May 31 '18 at 21:24
• Here's a picture of the kind of thing that can go wrong. In the diagram, ABCD is a polygon whose edge lines are all tangent to the circle. The polygon ABCD has an interior angle at C, which is actually an exterior angle of the kite OXCY. Of course, the problem here is that the points of tangency of the edge lines are not in the interiors of some of the edges; but your proof doesn't make any use of that fact. – Jack Lee May 31 '18 at 21:29
• @JackLee Oh, but that is not the definition of tangency. We do not define tangency of polygons by edge lines, but by the sides of the polygons, which are line segments. The problem with the picture is that it does not satisfy the hypothesis of the theorem that the polygon should have an incircle. Clearly, $ABCD$ is not a tangential polygon. I see that I had assumed that we were agreed on what it means for a polygon to be tangential. We were working with different understandings, I now see. In any case my other proof (unwittingly) avoids this. I consider it longer than this, though. – Allawonder May 31 '18 at 22:16