Does this ray fall inside this angle? I'm writing a routine to judge if an angle $\varphi$ in a plane lies in the smaller of the two angles formed by two other rays at angles $\varphi_1$ and $\varphi_2$. The angles aren't bounded in $\left[0,2\pi\right)$.
The usual convention is followed: the $0$ angle lies on the positive half of $x$ axis, and increasing angle goes counter-clockwise. In the diagram $\varphi$ lies in the convex angle while $\varphi'$ doesn't.

I'm thinking that since $\varphi$ is "between" $\varphi_1$ and $\varphi_2$, this relation has to hold
$$\sin(\varphi-\varphi_1)\sin(\varphi_2-\varphi)>0, \tag{1}$$
because the two sine values have the same sign. However, some angles in the concave side (the bigger angle) can also satisfy this by forming two obtuse angles, so (1) isn't good.
I would appreciate your ideas on a sound and simple mathematical method.
 A: Your data amount to the following: You are given two primary directions ${\bf w}_1:=(\cos\phi_1,\sin\phi_1)$ and ${\bf w}_2:=(\cos\phi_2,\sin\phi_2)$, whereby it is assumed that $\phi_1\ne\phi_2$ $(\pi)$, and an "arbitrary" direction ${\bf r}=(\cos\phi,\sin\phi)$. The direction ${\bf r}$ is called between ${\bf w}_1$ and ${\bf w}_2$ if $${\bf r}=\lambda_1 {\bf w}_1+\lambda_2{\bf w}_2\tag{1}$$ with $\lambda_1>0$ and $\lambda_2>0$. Solving $(1)$ for $\lambda_1$ and $\lambda_2$ gives
$$\lambda_1={\sin(\phi_2-\phi)\over\sin(\phi_2-\phi_1)},\qquad \lambda_2={\sin(\phi-\phi_1)\over\sin(\phi_2-\phi_1)}\ .$$
Therefore we obtain the following criterion: The direction ${\bf r}$ is  between ${\bf w}_1$ and ${\bf w}_2$ iff the three numbers
$$\sin(\phi_2-\phi), \quad \sin(\phi-\phi_1), \quad \sin(\phi_2-\phi_1)$$
all have the same sign.
If the ${\bf w}_i$ and ${\bf r}$ are given coordinatewise as
$${\bf w}_1=(u_1,v_1),\quad {\bf w}_2=(u_2,v_2),\quad {\bf r}=(x,y)$$
then the criterion is simple that the three determinants
$$v_2x-u_2y,\quad u_1y-v_1x,\quad u_1v_2-u_2v_1$$
should have the same sign.
A: We have three 2D vectors:
$$\begin{align}
\vec{u} &= (\cos \phi_1, \sin \phi_1) \\
\vec{v} &= (\cos \phi_2, \sin \phi_2) \\
\vec{p} &= (\cos \phi, \sin \phi) \end{align}$$
The 2D analog of cross product is zero if the vectors are parallel (or opposite), positive if the vectors are listed in counterclockwise order, and negative if the vectors are in clockwise order, with their angle always measured the short way round.
This means that if and only if the signs of
$$\begin{matrix}
\vec{u} \times \vec{v} &=& \cos\phi_1 \sin\phi_2 &-& \cos\phi_2 \sin\phi_1 &=& \sin(\phi_2 - \phi_1) \\
\vec{u} \times \vec{p} &=& \cos\phi_1 \sin\phi &-& \cos\phi \sin\phi_1 &=& \sin(\phi - \phi_1) \\
\vec{p} \times \vec{v} &=& \cos\phi \sin\phi_2 &-& \cos\phi_2 \sin\phi &=& \sin(\phi_2 - \phi)
\end{matrix}$$
are the same, then $\vec{p}$ is between $\vec{u}$ and $\vec{v}$, and correspondingly $\phi$ is between $\phi_1$ and $\phi_2$.
