Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I'm looking for the algorithm that determines the fact that a polygon has self intersection or hasn't. I'm not needed in calculation of the intersection points coordinates or how many intersection points there are.

share|cite|improve this question
The naïve algorithm is to compare each pair of edges to see whether they overlap. Do you want an optimal algorithm or have any interesting constraints (e.g. polygon endpoints on lattice points)? – Peter Taylor Nov 10 '11 at 9:10
No special constraints. I only need a very fast algorithm for this task. – MaxFX Nov 10 '11 at 10:05
Posssible duplicate of… – lhf Nov 10 '11 at 10:16
up vote 5 down vote accepted

There is the obvious algorithm of comparing all pairs of edges, which is $O(n^2)$ but probably is ok for small polygons. There is the Bentley–Ottmann algorithm sweep algorithm, which is $O(n \log n)$ but is harder to implement and probably only needed if $n$ is large. I think there is a $O(n)$ algorithm by Chazelle, which is very likely to be impractical.

In any case, note that you can test whether two line segments intersect without finding the intersection point. It's a simple matter of comparing signs of a few determinants. See

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.