Condition for the stable equilibrium of hemisphere having a particle attached at to its rim and standing over a sphere I have a heavy hemispherical shell of radius $a$ and a particle is attached to its rim at some point. The curved surface shell rests on a sphere of radius $b$ on its highest point.
I have to show that whatever be the weight of the particle, the equilibrium is stable if $\frac{b}{a} >\sqrt{5}-1$.
For the equilibrium to be stable, the combined centre of gravity of the shell and particle system should lie on the line joining the centre of sphere to the point of contact of two bodies i.e. highest point of the sphere.
Let the height of the combined C.G. be $h$ above the point of contact.
Then system will be in stable equilibrium if
$\frac{1}{h}>\frac{1}{a}+\frac{1}{b}$.
The C.G. of the shell is located at a distance of $\frac{3a}{8}$ from the centre of the shell i.e at  a distance $\frac{5a}{8}$ above the point of contact. Also the weight of the shell acts through the line joining the centre of sphere and point of contact. While the C.G. of the particle acts through a line parallel to this line.
I am having problem in finding the value of $h$. It is easier to find $h$ in cases where the masses of two components (shell and particle) act through the same line. But here the case is different.
Please suggest how to get $h$ in this case.
 A: When in equilibrium the plane containing the rim of the hemisphere makes an angle with the vertical, call it $\psi$, say.
The centre of the sphere, the point of contact $P$, and the centre of the hemisphere $O$ all lie in the same vertical line.
Let the particle, situated at point $A$, the lowest point of the rim, have mass $m$ and the hemisphere mass $M$. The centroid $G$ of the hemisphere is at a distance $\frac12a$ from $O$ on the axis of symmetry of the hemisphere.
Let the vertical line $OP$ intersect $AG$ at $B$, so that $h=BP$ the distance you seek. Let $\angle OAB=\alpha$, so $\tan\alpha=\frac12$.
Taking moments at $P$ we have $$mga\sin\psi=Mg \frac a2 \cos\psi\implies \tan\psi=\frac{M}{2m}$$
Then by the Sine Rule in $\triangle OAB$, we have $$\frac{a-h}{\sin\alpha}=\frac{a}{\sin(\alpha+\psi)}$$
Using the compound angle identity and rearranging, you get $$h=\frac{a\cos\psi+2a\sin\psi-a}{\cos\psi+2\sin\psi}$$
If you now make the assumption (which you have not justified) that if $\frac1h>\frac1a+\frac1b$, there is stable equilibrium, substituting for $h$ into this inequality leads to $$\frac ba>\cos\psi+2\sin\psi-1$$
Since we can write $\cos\psi+2\sin\psi$ in the form $\sqrt{5}\sin(\psi+\epsilon)$ which has a maximum value of $\sqrt{5}$ for all $\psi$, which is determined by the masses, then we can deduce the result required.

On the other hand, if you were to do this problem without making the assumption mentioned you would have to do the following:
Allow the hemisphere to roll without slipping through a small angle $\phi$ from the position of equilibrium. The point of contact $P'$ is now such that, if $C$ is the centre of the sphere, $OP'C$ is a straight line and the angle which $OP'C$ now makes with the vertical is $\theta$, and these two angles are related by the equation $$b\theta=a\phi$$
The vertical height of $O$ above $C$ is now $$(a+b)\cos\theta$$
The vertical depth of $A$ below $O$ is now $$a\cos(\theta+\phi+\psi)$$
The vertical depth of $G$ below $O$ is now $$\frac a2\sin(\theta+\phi+\psi)$$
With this information it is straightforward to write down an expression, after eliminating $\phi$, for the potential energy of the system $V$, as a function of $\theta$, taking fixed point $C$ as the zero level.
If you differentiate this once, it is easy to verify that at $\theta=0$, we get $\frac{dV}{d\theta}=0$ thus indicating equilibrium at $\theta=0$ as expected.
To establish the required inequality, you would then set $$\frac{d^2V}{d\theta^2}>0$$ since this is the requirement for stable equilibrium, and sure enough, after a bit of algebra, you arrive at $$\frac ba>\cos\psi+2\sin\psi-1,$$ and hence the result.
You might like to try this yourself.
