# If a polyhedron's faces and vertex figures are convex, is the polyhedron convex?

Suppose a polyhedron's faces are convex polygons, and its vertex figures are convex spherical polygons (or convex cones, depending on definitions). Must the polyhedron be convex?

As a counter-example for the analogous question in 2D, the pentagram $$\{5/2\}$$ has convex "faces" (line segments) and convex vertex figures ($$36^\circ$$ arcs; less than $$180^\circ$$) but is a non-convex, self-intersecting polygon.

The polyhedron may be in 3D Euclidean, spheric, or hyperbolic space. -- Actually it doesn't matter, because these all have standard embeddings in 4D:

$$\mathbb E^3=\{(x,y,z,w)\in\mathbb R^4\mid w=1\}$$ $$\mathbb S^3=\{(x,y,z,w)\in\mathbb R^4\mid x^2+y^2+z^2+w^2=1\}$$ $$\mathbb H^3=\{(x,y,z,w)\in\mathbb R^4\mid x^2+y^2+z^2-w^2=-1,\;w>0\}$$

The polyhedron is represented as a cone (or its intersection with the hypersurface). A point in 3D is represented as a ray, all the positive scalar multiples of some vector in $$\mathbb R^4$$. So we might as well work in $$\mathbb S^3$$, and use the standard dot product, though the following could be formulated in terms of general linear algebra (e.g. using quotient spaces instead of orthogonal complements).

Let $$A$$ be an abstract 3-polytope, and $$A_k$$ be the set of rank $$k$$ elements of $$A$$, for $$k=-1,0,1,2,3$$.

Let $$\sigma$$ be a function on $$A$$ with the following properties:

• If $$x\in A_k$$, then $$\sigma x$$ is a $$(k+1)$$-dimensional linear subspace of $$\mathbb R^4$$. (If you prefer, consider its intersection with $$\mathbb S^3$$, which is then a $$k$$-dimensional geodesic subspace.)
• If $$x\leq y$$ in $$A$$, then $$\sigma x\subseteq\sigma y$$.

This function is a realization of $$A$$. (It's nondegenerate if the last part is strengthened: $$x\leq y$$ if and only if $$\sigma x\subseteq\sigma y$$.)

Let $$v$$ be a function on incident pairs in $$A$$ (differing in rank by $$1$$) with the following properties:

• If $$A_k\ni x, then $$v(x,y)\in\mathbb S^3$$ is a unit vector orthogonal to $$\sigma x$$ and contained in $$\sigma y$$. Since the dimensions differ by $$1$$, there are only $$2$$ such unit vectors, differing in sign. (This vector $$v(x,y)$$ is supposed to point from $$\sigma x$$ to the "inside" of $$\sigma y$$, and $$-v(x,y)$$ to the "outside". These notions make sense for convex polytopes, and less sense for self-intersecting polytopes.)
• If $$A_{k-1}\ni w and $$x\neq y$$ and $$w, then the exterior product $$v(w,x)\wedge v(x,z)=-v(w,y)\wedge v(y,z)$$. (Note that $$v(w,x)\cdot v(x,z)=0$$ by the previous property.) Both of these unit bivectors are contained in the $$1$$-dimensional space $$\bigwedge^2\big(\sigma w^\perp\cap\sigma z\big)$$, so they must be equal up to sign. We require opposite signs. (If going from vertex $$w$$ to edge $$x$$ to face $$z$$ is turning clockwise, then going from vertex $$w$$ to the other edge $$y$$ to face $$z$$ should be turning anticlockwise.)

This function gives a kind of orientation of the polyhedron. It also gives the angles in faces, and the dihedral angles. Suppose $$w and $$w are two edges meeting at a vertex in a face; then the angle $$\theta$$ between the edges is completely determined (modulo $$2\pi$$) by $$\cos\theta=v(w,x)\cdot v(w,y)$$ and $$\sin\theta=v(w,x)\cdot v(y,z)$$.

In fact, given $$A$$, and the angles and edge lengths, we can reconstruct $$\sigma$$ and $$v$$, up to isometry of $$\mathbb S^3$$. Likewise, given $$A$$ and $$v$$, we can reconstruct $$\sigma$$.

All of that is just my idea of an (oriented) not-necessarily-convex polyhedron in spheric space. (Of course it could be modified for Euclidean or hyperbolic space. But those modifications only involve the metrical properties, not the incidence relations or convexity.)

Now to formalize the question. I'll use the notation $$[x,z]_k=\{y\in A_k\mid x\leq y\leq z\}$$.

Edges are convex:

$$\forall w\in A_{-1},\;\forall z\in A_1,\;\forall x\in[w,z]_0,\;\forall y\in[w,z]_0, \\ (y=x)\lor\big(v(x,z)\cdot v(w,y)>0\big).$$

Angles in faces are convex:

$$\forall w\in A_0,\;\forall z\in A_2,\;\forall x\in[w,z]_1,\;\forall y\in[w,z]_1, \\ (y=x)\lor\big(v(x,z)\cdot v(w,y)>0\big).$$

Dihedral angles are convex:

$$\forall w\in A_1,\;\forall z\in A_3,\;\forall x\in[w,z]_2,\;\forall y\in[w,z]_2, \\ (y=x)\lor\big(v(x,z)\cdot v(w,y)>0\big).$$

Faces are convex:

$$\forall w\in A_{-1},\;\forall z\in A_2,\;\forall x\in[w,z]_1,\;\forall y\in[w,z]_0, \\ (y\leq x)\lor\big(v(x,z)\cdot v(w,y)>0\big)$$

(in addition to edge and angle convexity for the face $$z$$; but we've already specified edge and angle convexity for everything).

Vertex figures are convex:

$$\forall w\in A_0,\;\forall z\in A_3,\;\forall x\in[w,z]_2,\;\forall y\in[w,z]_1, \\ (y\leq x)\lor\big(v(x,z)\cdot v(w,y)>0\big).$$

Polyhedron is convex?

$$\forall w\in A_{-1},\;\forall z\in A_3,\;\forall x\in[w,z]_2,\;\forall y\in[w,z]_0, \\ (y\leq x)\lor\big(v(x,z)\cdot v(w,y)>0\big)$$

In words, a polyhedron is convex, if each vertex is inside the half-space for each face (unless the vertex is part of the face), and the faces and vertex figures are convex polygons.

I haven't shown that this definition of a convex polyhedron is equivalent to the usual definitions (intersection of a finite set of half-spaces, convex hull of a finite set of points in 3D, or conical hull of a finite set of vectors in 4D). At least, I have shown it to myself. If someone wants to ask a new Question about the equivalence, I suppose I could answer. For now, take it for granted.

Here's an idea for a proof, in the case of Euclidean space.

An element $$x\in A_k$$ is realized as a $$k$$-dimensional affine subspace $$\sigma x\subset\mathbb E^3\cong\mathbb R^3$$. Let's ignore the distinction between a $$0$$-dimensional subspace and the unique point it contains, so that a vertex $$x\in A_0$$ has $$\sigma x\in\mathbb E^3$$ rather than $$\sigma x\subset\mathbb E^3$$.

Let $$l\in A_{-1}$$ and $$g\in A_3$$ be the least and greatest elements. Take an arbitrary face $$f$$ of the polyhedron, with inward normal vector $$v(f,g)$$. Let $$c\in\mathbb R$$ be the constant such that $$v(f,g)\cdot\sigma x=c$$ for all vertices $$x\leq f$$. We want to show that $$v(f,g)\cdot\sigma x>c$$ for all other vertices $$x\not\leq f$$.

Certainly $$v(f,g)\cdot\sigma y>c$$ for any vertex $$y$$ adjacent to a vertex $$x\leq f$$. Since the vertex figures are convex, we have $$v(f,g)\cdot v(x,e)>0$$ for any edge $$x\leq e\not\leq f$$. The adjacent vertex is $$\sigma y=\sigma x+t\,v(x,e)$$, where $$t>0$$ is the length of the edge; thus $$v(f,g)\cdot\sigma y=v(f,g)\cdot\sigma x+t\,v(f,g)\cdot v(x,e)>c$$. By similar reasoning, given $$x\leq f$$, any vertex $$y$$ in a face containing $$x$$ also has $$v(f,g)\cdot\sigma y>c$$, because $$\sigma y$$ is in the (2D) convex cone generated by two edges $$e,e'\geq x$$.

So we have, at least, a subset $$B\subseteq A$$ of the polyhedron, where all vertices are known to be either in $$f$$ or on the positive side of $$f$$. Consider $$B$$ as a surface, with a curve as its boundary. Take the vertex $$x$$ on the boundary of $$B$$ with the smallest value of $$v(f,g)\cdot\sigma x$$. It follows that $$v(f,g)\cdot v(x,e)\geq0$$ for any edge $$e\geq x$$ on the boundary, as that edge connects $$x$$ to another vertex on the boundary which must not have a smaller dot product with $$v(f,g)$$. Add to $$B$$ the faces incident with $$x$$, along with their edges and vertices, thus changing the boundary of $$B$$ so that it no longer contains $$x$$. It should be the case that any such new vertex $$y$$ has $$v(f,g)\cdot\sigma y\geq v(f,g)\cdot\sigma x$$, as a result of the convexity of the vertex figure at $$x$$ (but I'm not sure about this). Repeat until $$B=A$$. (Since the polyhedron is connected, every vertex will be reached.) The dot product with $$v(f,g)$$ is always increasing, thus always greater than $$c$$.

Can we fill in the gaps in this proof? Can it be modified for spheric space?

• It's very unclear what you are asking. Without rigorous definitions, there isn't much one can say. May 5 at 20:30
• The answer is surely Yes. This may be in Ziegler's Lectures on Polytopes. If you convert the convex cones to planes, then you should get the polyhedron as an intersection of halfspaces. May 9 at 13:39
• @JosephO'Rourke - What makes that work in 3D but not in 2D? May 9 at 18:57
• @mr_e_man: I think I cannot help, as you are seeking to exclude possibilities I myself don't see. May 9 at 20:04
• @JosephO'Rourke - I'm not sure I understand. Are you saying that it should in fact work in 2D? Again I'll point to star polygons. Perhaps Ziegler and you consider a polytope as merely a subset of space (rather than some other structure such as a collection of subspaces), thus disallowing self-intersection.... May 9 at 22:23

The OP doesn't specify that the abstract polyhedron must be a finite set.

So here's an infinite counter-example: a pentagrammic cylinder, or a stack of pentagrammic prisms, in hyperbolic space.

Using hyperbolic cylindrical coordinates, the vertices are located at

$$x_{(m,n)}=(\sinh(a)\cos(2m\cdot72^\circ),\sinh(a)\sin(2m\cdot72^\circ),\cosh(a)\sinh(nb),\cosh(a)\cosh(nb)),$$ $$m\in\mathbb Z\bmod5,\quad n\in\mathbb Z.$$

($$a$$ is the cylinder's radius. $$b$$ is a translation distance along the cylinder's axis.)

The polyhedron's faces are "rectangles", with vertex sets of the form $$\{x_{(m,n)},x_{(m,n+1)},x_{(m+1,n)},x_{(m+1,n+1)}\}$$. These are convex. (The angles in a face are less than $$90^\circ$$, because of the hyperbolic geometry. We need a word to replace "rectangle" which refers to symmetry rather than angles.)

The polyhedron's vertex figures are also convex, being rhombic. (In Euclidean space, one type of edge would have a dihedral angle of $$180^\circ$$, violating strict convexity, but in hyperbolic space it's less than $$180^\circ$$.)

The polyhedron as a whole is not convex, because the plane containing $$\{x_{(0,0)},x_{(0,1)},x_{(1,0)},x_{(1,1)}\}$$ has some vertices on both sides; e.g. $$x_{(2,0)}$$ and $$x_{(3,0)}$$ are separated by that plane.

Of course this could be projected to Euclidean space (Beltrami-Klein model; divide the first three coordinates by the fourth), but then it would have less symmetry, and two accumulation points $$(0,0,\pm1)$$.

I still think the answer is affirmative if the polyhedron is finite, but I haven't found a proof.