How to find the intersection of two circles on a sphere? I have two vectors $a, b \in \mathbb{R}^3$ which define points on the unit sphere, so $|a| = |b| = 1$. I also have two angles $\alpha, \beta \in [0, \frac{\pi}{2})$.
Taken together, $(a, \alpha)$ defines a circle $A$ of all vectors on the unit sphere that make an angle $\alpha$ with the center vector $a$, and analogously for $(b, \beta)$:
$$A = \{ p \in \mathbb{R}^3 \mid p \cdot a = \cos \alpha \land |p| = 1 \}$$
$$B = \{ p \in \mathbb{R}^3 \mid p \cdot b = \cos \beta \land |p| = 1 \}$$
Assuming that these circles intersect in exactly two points, how do I find these points?
The equations are quite simple:
$$a \cdot p = \cos \alpha$$
$$b \cdot p = \cos \beta$$
$$|p| = 1$$
I could try decomposing these into their individual coordinates and solving the system by hand, but I'm thinking there must be a more elegant approach involving (mostly) vector operations. Probably I should parameterize $p$ cleverly as a function of some $x$, such that I get a quadratic equation in $x$, because we have at most 2 solutions in non-degenerate cases.
 A: Found a way!
Consider point $q$, halfway between both solution points $p_1$ and $p_2$. The great circle defined by $p, q$ is perpendicular to the one defined by $a, b$.
We can find $q$ using the Pythagorean theorem on a sphere: $\cos A + \cos B = \cos C$. We apply it to triangle $a, q, p$ to find:
$$\cos aq \cos pq = \cos \alpha$$
Since the dot product gives the cosine of the angle:
$$(a \cdot q)(p \cdot q) = \cos \alpha$$
$$p \cdot q = \frac{\cos \alpha}{a \cdot q}$$
And the same for triangle $b, q, p$:
$$p \cdot q = \frac{\cos \beta}{b \cdot q}$$
Both are equal to $p \cdot q$, so we find:
$$(a \cdot q) \cos \beta = (b \cdot q) \cos \alpha$$
Rearrange the terms:
$$q \cdot (a \cos \beta - b \cos \alpha) = 0$$
So $q$ is perpendicular to the vector $a \cos \beta - b \cos \alpha$. And because $q$ is on the great circle defined by $a, b$, it is also perpendicular to $a \times b$, so we can find $q$ by crossing those two vectors:
$$q = \mathrm{normalize}((a \times b) \times (a \cos \beta - b \cos \alpha))$$
Now all we need to do is rotate $q$ perpendicular to the great circle $a, b$ towards $p$. The rotation axis $r$ is on the great circle $a, b$, so perpendicular to $a \times b$, and also perpendicular to $q$:
$$r = (a \times b) \times q$$
What is the angle $\rho$ over which we need to rotate $q$ to find $p$? We already have two nice equations for $p \cdot q$, so we just take the inverse cosine:
$$\rho = \arccos(p \cdot q) = \arccos\left(\frac{\cos \alpha}{a \cdot q}\right)$$
Finally, rotate $q$ around $r$ over an angle of $\rho$ or $-\rho$ to find our points $p$.
P.S. I'm doing this in performance-sensitive software, so I'm open to suggestions on how to do it with fewer calculations. Also, $\alpha$ and $\beta$ are rather small, leading to some loss of precision with all the cosines; ideas on how to avoid that are also welcome.
