Nested equilateral triangles Let triangle $ABC$ is an equilateral triangle. Triangle $DEF$ is also an equilateral triangle and it is inscribed in triangle $ABC \left(D\in BC,E\in AC,F\in AB\right)$. Find $\cos\measuredangle DEC$ if $AB:DF=8:5$.

Firstly, I would be very grateful if someone can explain to me how I am supposed to draw the diagram. Obviously I have made it by sight.
Let $\measuredangle DEC=\alpha$. We can note that $\triangle AEF \cong \triangle BFD \cong CDE$. This is something we can always use in such configuration. So $$AE=BF=CD, $$ $$AF=BD=CE.$$ Let $AB=BC=AC=8x$ and $DF=DE=EF=5x$. If we denote $CD=y,$ then $CE=AC-AE=AC-CD=8x-y$. Cosine rule on $CED$ gives $$25x^2=(8x-y)^2+y^2-2y\cos60^\circ(8x-y)$$ which is a homogenous equation. I got that $\dfrac{y}{x}=4\pm\sqrt{3}.$ Now using the sine rule on $CED$ $$\dfrac{CD}{DE}=\dfrac{\sin\alpha}{\sin60^\circ}\Rightarrow \sin\alpha=\dfrac{\sqrt{3}}{10}\cdot\dfrac{y}{x}=\dfrac{4\sqrt3\pm3}{10}.$$ Now we can use the trig identity $\sin^2x+\cos^2x=1$ but it doesn't seem very rational. Can you give me a hint? I was able to find $\sin\measuredangle DEC$ in acceptable way, but I can't find $\cos\measuredangle DEC$...
 A: Here I will show how you can draw the figure. The calculations that follow are somewhat similar to other answers.

We can build triangle $DEF$ by scaling down triangle $ABC$ and rotating it around its centroid $G$ by some angle $\alpha$.
Notice the hatched triangle $DHG$. This is a right triangle with sides:
$$GH = \frac13 AH =\frac{4}{12} AH$$
$$DG = \frac23 DY = \frac23 \times \frac58 AH = \frac{5}{12} AH $$
$$HD = \sqrt{DG^2 - GH^2} = \frac{3}{12} AH$$
It seems the designer of the problem intentionally used the ratio $\frac58$ to lead us to the $3-4-5$ right triangle!
Anyway, Now we are able to locate point $D$ on $BC$. It is at distance $\frac14 AH$ from point $H$ , the midpoint of BC. We can similarly locate points $E$ and $F$ , hence constructing triangle $DEF$.
Now, as for calculations, note that $\widehat{DEC}=120^o - \alpha$ and that $\widehat{DGH}=60^o - \alpha$. So $\widehat{DEC}=60^o+\widehat{DGH}$ . Therefore:
$$\cos \widehat{DEC} = \cos (60^o+\widehat{DGH}) = \cos60^o \cos \widehat{DGH} - \sin60^0 \sin \widehat{DGH}$$
$$= \frac12 \times \frac45 - \frac{\sqrt 3}{2} \times \frac35 = \frac{4-3\sqrt3}{10}$$
Note that when locating point $D$ we have two options, say, above and below $H$. The above calculations are for the option shown in the figure, where $D$ is selected below $H$. Alternatively, we could select $D$ to be above $H$, and respectively have $E$ below $K$. The interested reader can verify that in that case $\widehat{DEC}$ would be $\alpha$. And  $\; \cos\alpha \;$ can be calculated similar to above.
$$\cos \alpha = \cos (60^o-\widehat{DGH}) = \cos60^o \cos \widehat{DGH} + \sin60^0 \sin \widehat{DGH} = \frac{4+3\sqrt3}{10}$$
A: WLOG, $AB = 8, DF = 5$. Say $\angle DEC = \theta$
If $CD = x$ then $CE = 8 - x$
Applying sine law in $\triangle CDE$,
$\displaystyle \frac{\sin 60^0}{5} = \frac{\sin \theta}{x} = \frac{\sin(60^0+\theta)}{8-x}$
From first two, we have $\sin \theta = \frac{x \sqrt3}{10}$
From first and third, $\sin 60^0 \cos \theta + \cos 60^0 \sin\theta = \frac{4\sqrt3}{5} - \frac{x \sqrt3}{10}$
$\implies \cos\theta = \frac{8}{5} - \frac{3x}{10}$
Applying $sin^2\theta + \cos^2\theta = 1$, we get
$x^2 - 8x+\frac{39}{3} = 0 \implies x = 4 \pm \sqrt3$
So, $\cos\theta = \frac{4 \mp 3\sqrt3}{10}$ (please note $\pm$ for $x$ and corresponding $\mp$ for $\cos \theta$)
A: Note that there are two distinct solutions given the conditions in the problem.  This is because there are two admissible orientations of $\triangle DEF$:  one in which $AF > BF$, and one in which $AF < BF$.  We can understand this by noting that since $\triangle DEF$ and $\triangle ABC$ share the same center, the circumcircle of $\triangle DEF$ will intersect $\triangle ABC$ at six points, unless the side length of $\triangle DEF$ is less than or equal to half the side length of $\triangle ABC$ (in which case there will be either exactly three intersection points in the tangent case, or zero), or greater than or equal to the side length of $\triangle ABC$.
A simple way to solve for the required angle is to place the triangle on a coordinate system, say with $A = (0,0)$, $B = (8,0)$, $C = (4, 4 \sqrt{3})$, and let $F = (x,0)$ for some $0 \le x \le 8$.  Then $$E = \left(\frac{8-x}{2}, \frac{(8-x)\sqrt{3}}{2}\right).$$  We then require $EF = 5$, or $$25 = \left(x - \frac{8-x}{2}\right)^2 + \left(\frac{(8-x)\sqrt{3}}{2}\right)^2 = 3x^2 - 24x + 64.$$  Consequently $$x = 4 \pm \sqrt{3}$$ and depending on which root is chosen, $\angle DEC$ can be determined through a straightforward application of the Law of Sines.
A: You may calculate $\cos\alpha $ directly. Per the sine rule for $\triangle CDE$
$$\frac{\sin \alpha }{CD}= \frac{\sin (120-\alpha) }{AB-CD}= \frac{\sin 60 }{DE}
$$
Eliminate $CD$ to get
$$ \frac{AB}{DE}\sin 60 = \sin\alpha + \sin(120-\alpha)=2\sin 60\cos(60-\alpha)$$
which leads to $\cos(60-\alpha)=\frac45$. Then,
$$\cos\alpha = \cos(60\pm\cos^{-1}\frac45)
=\frac12\cdot \frac45\mp\frac{\sqrt3}2\cdot \frac35
=\frac{4\mp 3\sqrt3}{10}
$$
A: As others have done, I'll assume $AB=8,DF=5$ without loss of generality. Then equilateral triangles $\triangle ABC$ and $\triangle DEF$ have area $(\sqrt{3}/4)AF^2 = 16\sqrt{3}$ and $(\sqrt{3}/4)DF^2=(25/4)\sqrt{3}$ respectively. Since the triangles  $\triangle FAE,\triangle DBF, \triangle ECD$ are all congruent, they must each have area $$\frac{1}{3}\left(16\sqrt{3}-\frac{25}{4}\sqrt{3}\right)=\frac{13\sqrt{3}}{4}.$$ But each triangle has a 60-degree base angle, so we can also write the area of $\triangle FAE$ as
$$\frac12 AF\cdot AE \cdot \sin 60^\circ$$
Since $AE=AC-EC=8-AF$, this becomes $$\frac12 AF(8-AF)\frac{\sqrt{3}}{2}= \frac{13\sqrt{3}}{4}\implies AF(8-AF)=13$$
with $x=4\pm \sqrt{3}$ as solutions. As a minor variation on the other approaches, we may then use the law of cosines on triangle $\triangle CED$ to write
$$CD^2=CE^2+ED^2-2CE\cdot ED\cos\measuredangle DEC$$
and therefore
\begin{align}
\cos\measuredangle DEC
&=\frac{CE^2+ED^2-CD^2}{2CE\cdot ED}\\
&=\frac{(4\pm \sqrt{3})^2+5^2-(4\mp \sqrt{3})^2}{2(5)(4\mp \sqrt{3})}\cdot \frac{4\mp \sqrt{3}}{4\mp \sqrt{3}}\\
&=\frac{(25\pm 16\sqrt{3})(4\mp \sqrt{3})}{130}\\
&=\frac{52\pm 39\sqrt{3}}{130}\\
&=\frac{4\pm 3\sqrt{3}}{10}
\end{align}
as others have obtained.
