The General Forms:
Consider the above diagram. Let the radius of the cone be $r$ and its height be $h$.
Let the leftmost triangle be called $L$, rightmost triangle be called $R$.
Let the center $2$ triangles be $C_1$ and $C_2$ respectively from left to right.
Let $C_1$ and $C_2$ collectively be called $C$. (Check Note $5$, at the end)
Note:
- Cube inside $L$ is sitting on its face; in $C$, on edge; and in $R$, on a vertex.
- In $C_1, \ S$ touches the cone's lateral surface but since it is closer to us (remember it lies on a circle) so we don't see it that way from the front.
- $L,C,R$ are their respective best positions with respect to ease of observing them and maximizing the cube's side.
Cube's Side Length:
Diagram |
Similar Triangles |
$a$ in terms of $r$ and $h$ |
$L$ |
$\frac{r}{h}= \frac{\frac{a}{\sqrt{2}}}{h-a}$ |
$\frac{\sqrt{2}rh}{h+2\sqrt{r}}=a=a_L$ |
$C_1$ |
$\frac rh=\frac{\frac{\sqrt3a}2}{h-\frac a{\sqrt2}}$ |
$\frac{2rh}{\sqrt3h+\sqrt2r}=a=a_{C_1}$ |
$C_2$ |
$\frac rh=\frac{\frac a2}{h-a\sqrt2}$ |
$\frac{2rh}{h+2\sqrt2r}=a=a_{C_2}$ |
$R$ |
$\frac rh=\frac{a\sqrt{\frac23}}{h-\frac {2a}{\sqrt3}}$ |
$\frac{\sqrt3rh}{\sqrt2(h+\sqrt2r)}=a=a_R$ |
Note:
To arrive at the results as discussed above we use similar triangles as OP has mentioned.
From the diagram, the similar triangles are:
For $L: \Delta \text{ABQ}\sim\Delta \text{VBU}$,
For $C_1: \Delta \text{ABP}\sim\Delta \text{QBK}$,
For $C_2: \Delta \text{ABK}\sim\Delta \text{VBL}$,
For $R: \Delta \text{ABP}\sim\Delta \text{UBR}$
Results:
(Click the images to enlarge)
In descending order of the length of the side of the cube:
- From $O$ to $X$: $\rightarrow L>C>R$
- From $X$ to $Y\rightarrow L>R>C$
- From $Y$ to $Z\rightarrow L>C>R$
- From $Z$ to $\infty\rightarrow L>R>C$
where $O$ is origin and the value of $\frac hr$ at:
- $X= 2(\sqrt3-\sqrt2)\simeq0.636$
- $Y= \frac{2\sqrt6-4}{2\sqrt2-\sqrt3}\simeq0.820$
- $Z= \frac{4-\sqrt6}{3-2\sqrt2}\simeq9.037$
Note:
From $O(0)$ to $I(\simeq0.707)$, the side of the cube in $R$ is not given by the formula mentioned in the table. Because before $I$, $h<\frac r{\sqrt2}$, here, cube's length is restricted by the height of the cone. So, it's instead given by $a_R=\frac h{\sqrt3}$.
Similarly, in case of $C$, from $O$ to $J(\simeq1.932)$, $h<\frac{\sqrt2}{\sqrt3-1}r$, here, the cube's length is given by $C_2$ (as $C_1$ is not optimal here) and after $J$ its given by $C_1$ (as $C_2$ is not optimal here).
Conclusion:
Side of the largest possible cube inside a cone is always given by $\bf L$.
The only competition is between $R$ and $C$ for the $2^{nd}$ and $3^{rd}$ place.
Example:
Given, $h=12$ cms and $r=3\sqrt2$ cms. We know that the setting of L will give us the cube of largest side.
$\Rightarrow a_L=a=\frac{\sqrt2rh}{h+\sqrt2r}=\frac{\sqrt2⋅3\sqrt2⋅12}{12+\sqrt2⋅3\sqrt2}=12$
Thus, the side of the largest possible cube inside the given cone is $12$ cms.
Other Notes:
Note $\bf1$: (Why $\bf UV=a/\sqrt2$)
To know why $UV=a/\sqrt2\neq a/2$, check this post.
Similar reasoning is involved to calculate $KQ,LV$ and $RU$.
Note $\bf2$: (Diagram colors' meaning)
Cone's cross-section vertices: orange
Visible, hidden vertices of the cube: dark blue, light blue respectively
Visible, hidden edges of the cube: solid line, dotted line respectively
Imaginary lines/point to show the similar triangles we considered: red
Note $\bf3$: (Note of Thanks)
Special thanks to @David for this answer. It was especially helpful to figure out everything related to $R$.
Note $\bf4$: (Fun Fact)
In the given diagram, $h=3r$ and surprisingly, though it wasn't intended, the cubes inside look as if drawn to scale.
Just by observing, one can see that $a_L>(a_{C_1}\text{ & }a_{C_2})>a_R$ which is in accordance with our results.
Note $\bf5$: (Collectively called $\bf C$?)
Observe $C_1$ and then $C_2$. Notice anything? If not, then it would be best to take a dice and keep it on the table on $1$ edge and observe from front while rotating it.
You'll notice that $C_1$ and $C_2$ are same orientations (i.e. standing on an edge), we just rotated the cone along the $y$-axis passing through the vertex and the center of the base of the cone (that is, rotation along the red line, from the diagram).
The difference between $C_1$ and $C_2$ is that, in $C_1: SQUW$ touch the cone while in $C_2: VW$ touch the cone.
Note $\bf6$: (Note of Gratitude)
My heartiest thanks to @peterwhy for his immeasurable patience and for providing his great insights and proofreading each and every word and even diagrams! He helped me get clarity over various things too (and it wasn't easy).
Do check this post where he lucidly explains everything about $\bf R$.
Tl;dr: Side of the largest possible cube inside a cone is always given by $L$ (when the cube is sitting on its face).