Determine the concavity of an edge Hie guys while trying to determine the concavity of an edge I came across a post saying
 If na and nb are the normals of the both adjacent faces, and pa and pb vertices of the both faces that are not connected to the edge in question, wherein na and pa belongs to the face A, and nb and pb to the face B, then

( pb - pa ) . na <= 0 => ridge edge

as well as

( pb - pa ) . na > 0 => valley edge

where "ridge edge" denotes what you have named convex edge, and "valley edge" denotes what you've named concave edge.

You can avoid numerical inaccuracies in nearly flat connections to be classified if you compute the dot-product of the both normals and compare that with a defined threshold, and do the above decision only of the threshold is exceeded.

The problem is that I ( pb - pa ) . na is not equal to ( pb - pa ) . nb as he says it should be and so i still cant solve my problem.
please help
Thanks.
 A: If you have $n$ vertices $\vec{p}_1, \dots, \vec{p}_n$, considering them cyclically ($\vec{p}_0 = \vec{p}_n$ and $\vec{p}_{n+1} = \vec{p}_1$), and there are no overlapping or intersecting vertices or edges (i.e. it is a simple polygon), the polygon is convex if all $b_i$, $i = 1..n$ have the same sign (or are zero):
$$b_i = \left(\vec{p}_{i} - \vec{p}_{i-1}\right)\times\left(\vec{p}_{i+1} - \vec{p}_i\right)$$
If $\vec{p}_i = (x_i, y_i)$, then
$$b_i = (x_i - x_{i-1})(y_{i+1}-y_{i}) - (y_i - y_{i-1})(x_{i+1}-x_i)$$
Simply put, the sign of $b_i$ is the same as $\sin\theta_i$, where $\theta_i$ is the angle between the edges at vertex $i$ (if there are no zero-length edges). The interior angle at vertex $i$ is $180° - \theta_i$ if the polygon is drawn counterclockwise, and $180° + \theta_i$ if clockwise. (If the polygon is drawn counterclockwise, the left side of the vertex, looking along the edge arriving at the vertex, is the interior. If clockwise, then interior is on the right side, and we need to reverse the order of the two vectors to get the correct interior angle; which is the same as negating $\theta_i$.)
In two dimensions, the cross product of vectors $\vec{a}$ and $\vec{b}$ is $\vec{a} \times \vec{b} = \left\lVert\vec{a}\right\rVert \left\lVert\vec{b}\right\rVert\sin\theta$, where $\theta$ is the angle between the two vectors. Here, we utilize the properties of the $\sin\theta$ part. (And the fact that since there are no zero-length edges, then the product of the two vector lengths is always positive, $\left\lVert\vec{a}\right\rVert \left\lVert\vec{b}\right\rVert \gt 0$.) Therefore, $b_i = 0$ if the edge to vertex $i$ ($\vec{p}_i-\vec{p}_{i-1}$) is collinear with the edge from vertex $i$ ($\vec{p}_{i+1}-\vec{p}_i$). If $b_i \gt 0$, the edges turn counterclockwise at vertex $i$ (i.e., $\theta_i \lt 0$). If $b_i \lt 0$, the edges turn clockwise at vertex $i$ (i.e. $\theta_i \gt 0$).
The reason why testing the sines of the angles between successive edges is sufficient to determine if a simple polygon (with no zero-length edges and no intersections between edges or vertices) is convex or not, is because the sine changes sign for angles less than zero, or greater than $180°$. If $\sin\theta_i$ are all nonnegative, or are all nonpositive, then all interior angles are at most $180°$. If all $\sin\theta_i$ are positive, or all negative, then all interior angles are strictly less than $180°$.
