Suppose a plane quadrilateral ABCD without sides parallel to y-axis, and let $m_1, m_2, m_3, m_4$ be the slopes of the equations of sides AB, BC, CD, DA (the cartesian axes being orthogonal or oblique). Having made these definitions, we may state the following theorem:

ABCD is a concave quadrilateral iff $$(m_1-m_2)(m_2-m_3)(m_3-m_4)(m_4-m_1)<0$$

This theorem (which I found out on my own) can be proved by a reasoning which takes in consideration the position of the vertices of the quadrilateral in relation to the diagonals, but I would appreciate if someone could show me a different way of proving it.


Here we will prove a stronger theorem: Let $L_1=p_1x+q_1y+r_1=0$, $L_2=p_2x+q_2y+r_2=0$, $L_3=p_3x+q_3y+r_3=0$, $L_4=p_4x+q_4y+r_4=0$ be cartesian equations of sides AB, BC, CD, DA of a plane quadrilateral ABCD. Then we may state the following theorem:

ABCD is a concave quadrilateral iff $$\begin{vmatrix}p_1 & q_1\\p_2 & q_2\\\end{vmatrix}\begin{vmatrix}p_2 & q_2\\p_3 & q_3\\\end{vmatrix}\begin{vmatrix}p_3 & q_3\\p_4 & q_4\\\end{vmatrix}\begin{vmatrix}p_4 & q_4\\p_1 & q_1\\\end{vmatrix}<0$$

Let $A=(x_1,y_1)$, $B=(x_2,y_2)$, $C=(x_3,y_3)$, $D=(x_4,y_4)$ be the cartesian coordinates of the vertices of the quadrilateral.

Equations of the diagonal lines:

$$AC(x,y)\equiv\begin{vmatrix}x & y & 1\\x_1 & y_1 & 1\\x_3 & y_3 & 1\\\end{vmatrix}=0$$

$$BD(x,y)\equiv\begin{vmatrix}x & y & 1\\x_2 & y_2 & 1\\x_4 & y_4 & 1\\\end{vmatrix}=0$$

Equation of side AB:

$$AB(x,y)\equiv\begin{vmatrix}x & y & 1\\x_1 & y_1 & 1\\x_2 & y_2 & 1\\\end{vmatrix}=0,$$ $$AB(x,y)\equiv (y_1-y_2)x+(x_2-x_1)y+(x_1y_2-x_2y_1)=0$$

As this equation and the equation $L_1=p_1x+q_1y+r_1=0$ given above represent the same line, $\exists$ $k_1\neq 0$ such that $$(y_1-y_2)=k_1p_1, (x_2-x_1)=k_1q_1.$$

Likewise $\exists$ $k_2$, $k_3$, $k_4$, all different from zero, such that $$(y_2-y_3)=k_2p_2, (x_3-x_2)=k_2q_2.$$ $$(y_3-y_4)=k_3p_3, (x_4-x_3)=k_3q_3.$$ $$(y_4-y_1)=k_4p_4, (x_1-x_4)=k_4q_4.$$

That being done, we note that each diagonal line of a quadrilateral splits the plane in two half-planes, and how the respective remaining vertices locate relating to the diagonal line it's a key to classify the quadrilateral:

Lemma I. A plane quadrilateral is convex iff each of its diagonal lines splits the respective remaining vertices in distinct half-planes.

Lemma II. A plane quadrilateral is crossed iff each of its diagonal lines doesn't split the respective remaining vertices, both staying in the same half-plane.

Lemma III. A plane quadrilateral is concave iff one diagonal line splits the remaining vertices in distinct half-planes, and the other diagonal line doesn't split the remaining vertices.

Translating these purely geometric lemmas into the algebraic language of analytic geometry we get

I.ABCD is convex iff $AC(x_2,y_2).AC(x_4,y_4)<0$ and $BD(x_1,y_1).BD(x_3,y_3)<0$ .

II. ABCD is crossed iff $AC(x_2,y_2).AC(x_4,y_4)>0$ and $BD(x_1,y_1).BD(x_3,y_3)>0$.

III. ABCD is concave iff $AC(x_2,y_2).AC(x_4,y_4).BD(x_1,y_1).BD(x_3,y_3)<0$

Therefore ABCD is concave

$$\Leftrightarrow \begin{vmatrix}x_2 & y_2 & 1\\x_1 & y_1 & 1\\x_3 & y_3 & 1\\\end{vmatrix}\begin{vmatrix}x_4 & y_4 & 1\\x_1 & y_1 & 1\\x_3 & y_3 & 1\\\end{vmatrix}\begin{vmatrix}x_1 & y_1 & 1\\x_2 & y_2 & 1\\x_4 & y_4 & 1\\\end{vmatrix}\begin{vmatrix}x_3 & y_3 & 1\\x_2 & y_2 & 1\\x_4 & y_4 & 1\\\end{vmatrix}<0$$

$$\Leftrightarrow \begin{vmatrix}x_1-x_2 & y_1-y_2\\x_3-x_2 & y_3-y_2\\\end{vmatrix}\begin{vmatrix}x_1-x_4 & y_1-y_4\\x_3-x_4 & y_3-y_4\\\end{vmatrix}\begin{vmatrix}x_2-x_1 & y_2-y_1\\x_4-x_1 & y_4-y_1\\\end{vmatrix}\begin{vmatrix}x_2-x_3 & y_2-y_3\\x_4-x_3 & y_4-y_3\\\end{vmatrix}<0$$

$$\Leftrightarrow \begin{vmatrix}-k_1q_1 & k_1p_1 \\k_2q_2 & -k_2p_2\\\end{vmatrix}\begin{vmatrix}k_4q_4 & -k_4p_4 \\-k_3q_3 & k_3p_3 \\\end{vmatrix}\begin{vmatrix}k_1q_1 & -k_1p_1 \\-k_4q_4 & k_4p_4\\\end{vmatrix}\begin{vmatrix}-k_2q_2 & k_2p_2\\k_3q_3 & -k_3p_3\\\end{vmatrix}<0$$

$$\Leftrightarrow (k_1k_2k_3k_4)^2\begin{vmatrix}p_2 & q_2 \\p_1 & q_1\\\end{vmatrix}\begin{vmatrix}p_3 & q_3 \\p_4 & q_4\\\end{vmatrix}\begin{vmatrix}p_4 & q_4 \\p_1 & q_1\\\end{vmatrix}\begin{vmatrix}p_3 & q_3\\p_2 & q_2\\\end{vmatrix}<0$$

$$\Leftrightarrow \begin{vmatrix}p_1 & q_1 \\p_2 & q_2\\\end{vmatrix}\begin{vmatrix}p_2 & q_2 \\p_3 & q_3\\\end{vmatrix}\begin{vmatrix}p_3 & q_3 \\p_4 & q_4\\\end{vmatrix}\begin{vmatrix}p_4 & q_4\\p_1 & q_1\\\end{vmatrix}<0$$


Thus, if no side of ABCD is parallel to y-axis, ABCD is concave

$$\Leftrightarrow \begin{vmatrix}m_1 & -1\\m_2 & -1\\\end{vmatrix}\begin{vmatrix}m_2 & -1\\m_3 & -1\\\end{vmatrix}\begin{vmatrix}m_3 & -1\\m_4 & -1\\\end{vmatrix}\begin{vmatrix}m_4 & -1\\m_1 & -1\\\end{vmatrix}<0$$

$$\Leftrightarrow (m_1-m_2)(m_2-m_3)(m_3-m_4)(m_4-m_1)<0$$


Has anyone ever seen this theorem in a book, paper or in the internet before?

Is anyone acquainted with a different proof?

  • $\begingroup$ Using the setup in this answer, one can show that, for a quad with intersecting diagonals and side-slopes $p$, $q$, $r$, $s$, $$(p-q)(q-r)(r-s)(s-p) = \frac{abcd(a-c)^2(b-d)^2\sin^2(\theta-\phi)}{(\cdots)^2}$$ For a convex quad, we have $ac\leq 0$ and $bd\leq 0$, making the product non-negative. "All we need to do" is show that concavity implies $abcd<0$. (Note that we may assume one of $a$ and $c$, and one of $b$ and $d$ are positive. Therefore, $abcd<0$ is the condition that exactly one factor is negative.) Handle parallel diags separately. $\endgroup$ – Blue Oct 22 '18 at 15:44
  • $\begingroup$ Blue, have you ever seen this theorem in a book, paper or in the internet before? $\endgroup$ – MrDudulex Oct 22 '18 at 16:03
  • $\begingroup$ It wouldn't surprise me if this is in the literature somewhere. Lots of mathematicians pre-date us. :) This seems like it might've been known to Bretschneider or his contemporaries in the mid-1800s. (BTW, as I mentioned to another M.SE'er recently: The fact that someone else derived a result earlier doesn't diminish your own accomplishment. Steve Fisk called this kind of thing an "accident of time".) Even if your various slope formulas are known, a unifying concept (such as your circumscribed conics) could be novel and enlightening. $\endgroup$ – Blue Oct 22 '18 at 16:25
  • $\begingroup$ Yes, I also think that this theorem (and my slope formulas) are hidden in some obscure articles of mathematical journals of early-mid 1800s collecting dust since then in a library $\endgroup$ – MrDudulex Oct 22 '18 at 16:33
  • $\begingroup$ Try looking through old textbooks on Analytical Geometry; the cloth-bound books that look printed straight from a typewriter (with hand-written Greek letters!). Used bookstores sometimes have real gems. Even stuff as recent as the 1950s could be very heavy on wide variety of geometric topics. Nowadays, textbooks cover only lines and circles and conics, and only in a very superficial manner. Anyway, such textbooks might get you started on a comprehensive literature search that reaches further and further back in time. Reviving dusty results is itself a service to Mathematics. $\endgroup$ – Blue Oct 22 '18 at 17:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.