From Michigan State University's Herzog contest:

Problem 6, 1981

Three points are taken at random on a unit sphere. What is the probability that the area of the spherical triangle exceeds the area of a great circle?

I assume we always take the unique proper spherical triangle, with sides and angles not greater than $\pi$. I assume "the area of a great circle" is the plane area $\pi$, since a proper spherical triangle never has area larger than the spherical area $2 \pi$ on each side of a great circle. I assume the random distribution is uniform, such that the probability each point is within a given measurable subset of the sphere is proportional to the spherical area of that subset.

First, given triangle vertices $A,B,C$ on the unit sphere, we can rotate and/or mirror the coordinate system so that $C = (0,0,1)$, $A$ is on the $x \geq 0, y=0$ half-plane, and $B$ is on the $y \geq 0$ hemisphere. Label the side lengths opposite $A$, $B$, and $C$ as $a$, $b$, and $c$ respectively. Label the (dihedral) angles at $A$, $B$, and $C$ as $\alpha$, $\beta$, and $\gamma$ respectively. Ignore a degenerate triangle case with any of these variables exactly equal to $0$ or $\pi$; this happens with probability zero. Then all six variables are in the interval $\{a,b,c,\alpha,\beta,\gamma\} \subset (0,\pi)$. We can parameterize the probability distribution as:

$$ \begin{align*} P(a < a_0) &= \frac{1-\cos a_0}{2} \\ P(b < b_0) &= \frac{1-\cos b_0}{2} \\ P(\gamma < \gamma_0) &= \frac{\gamma_0}{\pi} \end{align*}$$

Variables $a$, $b$, and $\gamma$ are independent, so the overall probability density function is

$$ dP = \frac{1}{4\pi} \sin a \sin b\, da\, db\, d\gamma $$

The spherical area $\sigma$ of the triangle is

$$ \sigma = \alpha + \beta + \gamma - \pi $$

The outcome $\sigma > \pi$ is only possible if $\gamma > \frac{\pi}{2}$, since the spherical triangle covers a subset of the region between the planes containing $\{O,A,C\}$ and $\{O,A,B\}$ with area, $2 \gamma$. Similarly $\sigma > \pi$ implies $\alpha > \frac{\pi}{2}$ and $\beta > \frac{\pi}{2}$.

We also have Napier's analogy

$$ \tan \frac{\alpha+\beta}{2} = \frac{\cos \frac{a-b}{2}}{\cos \frac{a+b}{2}} \cot \frac{\gamma}{2} $$

Since $\{a,b\} \subset (0, \pi)$, $|a-b|<\pi$ and $\cos \frac{a-b}{2} > 0$. If $\sigma > \pi$ then $\{\alpha, \beta, \gamma\} \subset (\frac{\pi}{2}, \pi)$ which implies $\tan \frac{\alpha+\beta}{2} < 0$ and $\cot \frac{\gamma}{2} > 0$. Therefore $\cos \frac{a+b}{2} < 0$, and $\frac{a+b}{2} > \frac{\pi}{2}$.

So given $\{\alpha, \beta, \gamma, \frac{a+b}{2}\} \subset (\frac{\pi}{2}, \pi)$, all these inequalities are equivalent:

$$ \begin{align*} \sigma &> \pi \\ \alpha + \beta + \gamma &> 2\pi \\ \pi - \frac{\gamma}{2} &< \frac{\alpha+\beta}{2} \\ \tan \left(\pi - \frac{\gamma}{2}\right) &< \tan \frac{\alpha+\beta}{2} \\ -\tan \frac{\gamma}{2} &< \frac{\cos \frac{a-b}{2}}{\cos \frac{a+b}{2}} \cot \frac{\gamma}{2} \\ \tan^2 \frac{\gamma}{2} &> - \cos \frac{a-b}{2}\, \sec \frac{a+b}{2} \\ \gamma &> 2\tan^{-1} \sqrt{- \cos \frac{a-b}{2}\, \sec \frac{a+b}{2}} = \gamma_L(a,b) \end{align*}$$

where the function $\gamma_L(a,b)$ is defined where $0 \leq a \leq \pi$, $0 \leq b \leq \pi$, and $a+b>\pi$. The probability in question is

\begin{align}P(\sigma > \pi) &= \int_0^\pi \int_{\pi-a}^\pi \int_{\gamma_L(a,b)}^\pi \frac{1}{4\pi} \sin a \sin b\, d\gamma\, db\, da\\&= \frac{1}{4\pi} \int_0^\pi \int_{\pi-a}^\pi [\pi - \gamma_L(a,b)] \sin a \sin b\, db\, da\end{align}

I've found we can also write

$$ \gamma_L(a,b) = \pi - \cos^{-1}\left(\cot \frac{a}{2} \cot \frac{b}{2}\right) $$

Either way, integration by parts can get rid of the inverse trig function in the integrand, leaving a square root, but I haven't been able to make much progress beyond that in solving the definite integral.

Wolfram Alpha gives the numeric answer as apparently $\frac{1}{6}$ (1) (2). But it seems it can't solve it symbolically either.

Given this was in a limited-time competition, maybe there's some other simpler way to go about this. Some sort of symmetry grouping related points or triangles? How can we prove the exact probability?

  • $\begingroup$ isn't it kind of begging for the half-tangent substitution? Otherwise, I would express things like $\arccos, \arctan $ etc with its complex representation and attempt a contour integral. $\endgroup$
    – dezdichado
    Dec 1, 2021 at 0:39
  • 1
    $\begingroup$ I think the area of a great circle is the $2\pi$ solid angle represented by either side. $\endgroup$ Dec 1, 2021 at 1:13
  • $\begingroup$ for what it's worth, after half-tangent substitutions, then integral is: $$\int_0^\infty\int_0^{1/x}\dfrac{4\arccos(xy)xy}{\pi(1+x^2)^2(1+y^2)^2}dxdy = \dfrac{1}{6}$$ also confirmed by calculator. $\endgroup$
    – dezdichado
    Dec 1, 2021 at 2:21
  • $\begingroup$ @RossMillikan That would mean the probability is zero; not an extremely interesting question. $\endgroup$
    – aschepler
    Dec 1, 2021 at 2:57
  • $\begingroup$ @ascheplerOK I see that. $\endgroup$ Dec 1, 2021 at 3:04

1 Answer 1


The substitution $u=\cot(a/2)\cot(b/2)$ yields $$I(a)=\int_{\pi-a}^\pi\arccos\left(\cot\frac a2\cot\frac b2\right)\sin b\,db=\int_0^1\frac{4u\tan^2(a/2)\arccos u}{(1+u^2\tan^2(a/2))^2}\,du$$ so that $$P(\sigma>\pi)=\frac1{4\pi}\int_0^\pi I(a)\sin a\,da=\frac4\pi\int_0^{\pi/2}J(a)\sin^2a\tan a\,da$$ where $\displaystyle J(a)=\int_0^1\frac{u\arccos u}{(1+u^2\tan^2a)^2}\,du$. Integration by parts yields \begin{align}J(a)&=-\frac{\arccos u}{2(1+u^2\tan^2a)\tan^2a}\bigg\vert_0^1-\frac1{2\tan^2a}\int_0^1\frac{du}{(1+u^2\tan^2a)\sqrt{1-u^2}}\\&=\frac\pi{4\tan^2a}-\frac1{2\tan^2a}\cdot\frac\pi{2\sqrt{\tan^2a+1}}\\&=\frac{\pi(1-\cos a)}{4\tan^2a}\end{align} where the last integral is evaluated by taking $t=u/\sqrt{1-u^2}$. Therefore, \begin{align}P(\sigma>\pi)=\int_0^{\pi/2}(1-\cos a)\sin a\cos a\,da=\frac16.\end{align} Not sure if there is a quicker geometrical answer.


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .