Finding the largest equilateral triangle inside a given triangle My wife came up with the following problem while we were making some decorations for our baby: given a triangle, what is the largest equilateral triangle that can be inscribed in it? (In other words: given a triangular piece of cardboard, what is the largest equilateral triangle that you can cut out of it?)
She also came up with the following heuristic/conjecture: take the largest angle of the given triangle. (It is guaranteed to be at least $60^\circ = \pi/3$, as not all three angles of a triangle can be less than that.) Now the answer (the largest inscribed equilateral triangle) can be found among those made by marking off a $60^\circ$ angle at that vertex, with the two ends chosen somehow. (In other words: the inscribed equilateral triangle can be chosen to have that vertex as one of its vertices.)

My intuition for geometry is not so good. :-) I played with a few examples in Geogebra and couldn't find any counterexamples, nor could I think of a proof, so I'm asking here.
This is similar to Find the maximum area possible of equilateral triangle that inside the given square and a special case of Largest Equilateral Triangle in a Polygon (whose paper I don't have access to—all I could find via citations to that paper is that it gives an $O(n^3)$ algorithm—and in any case the problem may be simpler for a triangle).

Questions:


*

*How can one find the largest equilateral triangle that can be inscribed in a given triangle?

*Can such a triangle always be found with one vertex at one of the vertices of the given triangle, specifically the one with the largest angle?

*(If the answer to the above is no) When is the above true? For instance, is the conjecture true when the triangle is isosceles, with the two sides adjacent to the largest angle equal?

 A: Let $T$ be the given triangle, having vertices $A_i$, angles $\alpha_i$, and edges   $a_i=[A_{i-1},A_{i+1}]$. Use $\triangle$ as variable for equilateral triangles contained in $T$, and denote the vertices of such triangles by $v_j$. Given an instance $I:=(\triangle, T)$ call the number of incidences $v_j\in a_i$ the defect of this instance. Then one has
Lemma. If the defect of the instance $I$ is $\leq3$ then $\triangle$ is not maximal.
Proof. We leave defects $\leq2$ to the reader and bother about instances with defect $3$. If $v_1=A_1$ is a vertex of $T$, $v_2$ lies on the adjacent edge $a_2$, and $v_3$ lies in $T^\circ$ then one can just move $v_2$ along $a_2$ to increase $\triangle$. If $v_1=A_1$  and $v_2$ lies on the opposite edge $a_1$ then one can slightly rotate $\triangle$ around $v_1$, thereby freeing $v_2$, so that $\triangle$ then can be enlarged by scaling from $v_1$. If $v_1$, $v_2$ are lying on the same edge $a_j$ and $v_3$ on some other edge then one may slightly translate $[v_1,v_2]$ along $a_j$ to free $v_3$.
It remains the case that the three vertices $v_i$ of $\triangle$ are lying on the three edges $a_i$ of $T$. Assume that $\triangle$ has side length $s$, and assume that $\alpha_3\leq60^\circ$. If we let $v_1$ and $v_2$ glide along $a_1$ and $a_2$ such that $|v_1v_2|=s$ at all times then $v_3$ will describe an arc of an ellipse (some computation is needed to verify this). The edge $a_3$ is intersecting or touching this arc at the initial position of $v_3$. It follows that we can free $v_3$ in this way and then enlarge $\triangle$.
We therefore have to consider only instances $I$ with a defect of $4$ or more (the latter can happen if  one or three of the $\alpha_i$ are $=60^\circ$). A defect $\geq4$ can be realized in the following ways:
(i) The vertex $v_1$ of $\triangle$ coincides with the vertex $A_1$ of $T$, and the vertices $v_2$, $v_3$ are lying on the opposite edge $a_1$ of $T$.  This is possible only if $\alpha_2$ and $\alpha_3$ are $\leq60^\circ$.
(ii) The vertex $v_1$ of $\triangle$ coincides with the vertex $A_1$ of $T$, the vertex $v_2$ is on  the adjacent edge $a_2$ or $a_3$ of $T$, and the vertex $v_3$ is on the opposite edge $a_1$ of $T$.  This is possible only if $\alpha_1\geq60^\circ$.
(iii) The vertices $v_1$, $v_2$ of $\triangle$ coincide with the two vertices $A_1$, $A_2$ of $T$. This is possible only if  $\alpha_1$ and $\alpha_2$ are $\geq60^\circ$.
In this way the problem has been reduced to the (computational) analysis of a finite number of cases. 
If $T$ has two angles $<60^\circ$ then we have to compare the three colored $\triangle$s in the following figure. It is obvious that the red triangle is the largest.

If $T$ has two angles $>60^\circ$ then we have to compare the three colored triangles in the following figure. Which of them is the largest depends in an intricate way on the angles $\alpha$ and $\beta$. Assume $\beta>\alpha$, as in the figure. Then  the green $\triangle$ is larger than the blue $\triangle$. If $\alpha+2\beta>240^\circ$ then the green $\triangle$ is also larger than the red $\triangle$, hence the largest of the three. If $\alpha+2\beta<240^\circ$ then the red triangle is the largest. The following surprise is easy to verify: If $T$ has angles $80^\circ$, $80^\circ$ and $20^\circ$ then $s=u=v$. It follows that in this case there are three different maximal $\triangle$s in $T$.

To substantiate the claims made in the last paragraph note that the angles $\eta$ in the figure are $=240^\circ-\alpha-\beta$. The sine theorem then gives
$$u=\sin\beta{s\over\sin\eta}=s{\sin\beta\over\sin(240^\circ-\alpha-\beta)},\quad v=s{\sin\alpha\over\sin(240^\circ-\alpha-\beta)}\ .$$
It follows that
$$\max\{s,u,v\}={s\over\sin(240^\circ-\alpha-\beta)}\max\{\sin(240^\circ-\alpha-\beta),\sin\beta,\sin\alpha\}\ ,$$
so that it remains to decide which of the last three entries is largest.
A: Take the two vertices on the shortest side and draw circles from each to the other:

If the circles intersect within the triangle as above, the largest equilateral is as shown in green. It can be seen that if the green triangle is rotated around either vertex, it will have to be smaller.
If the circles don't intersect within the triangle as in the following case:

...then draw a perpendicular from the vertex opposite the longest side and form the equilateral as shown:

In that case it can also be seen that if the green triangle is rotated around the vertex it would have to be smaller.
EDIT: One more case where the inscribed triangle occupies none of the vertices

But that isn't optimal since the triangle can be rotated to afford more room.
EDIT #2: (reply to @ran)

A: Example 1



*

*On the longest side AC, draw circles of radius AC units at both ends.

2. Mark D, the intersection point of the two circles, and draw intervals DA and DC.



*Construct a line parallel to DC through B, and mark E, the point of intersection with AC. Then construct a line parallel to DA through B, and mark F, the point of intersection with AC



*Shade $\triangle$BEF, the required triangle.


Example 2



*

*Repeat steps 1 and 2 above. Since AD passes through the triangle, mark its intersection with BC in E. Draw a line $\parallel$ to CD through E, and mark its intersection with AC in F.



*Shade $\triangle$AEF, the required triangle.


The proofs are elementary: corresponding angles between parallel lines are equal.
A: My answer is not a formal one and is mainly based on qualitative reasoning and some goniometry. It follows the remarks of @Christian Blatter and tries to complete them.
The generic triangle $T$ on which to inscribe the maximum equilateral triangle $T_e$ (whose side we'll call $w$) can be uniquely specified by the lenght $c$ of one side $AB$ and by the adjacent angles to this side 
$\alpha$ and $\beta$ (see figure). Naturally it must be $\alpha+\beta\le 180^\circ$

Next, let's name the angles $\alpha, \beta, \gamma$ such that $\alpha \ge \beta \ge \gamma$. The $T$ vertices will be named with the usual convention in which $A$ is the vertex whtih angle $\alpha$, $B$ is the vertex whtih angle $\beta$ and $C$ is the vertex whtih angle $\gamma$, while the sides $a,b,c$ are the ones opposite the vertices $A,B,C$
Then let's suppose one vertex of $T_e$ is coincident with one vertex of $T$ (we'll verify if this assumption is optimum later)
There are two main cases for the ampliude of the angles:
(I)    $ \alpha,\beta > 60^\circ, \quad  \gamma < 60^\circ$
and
(II)    $\alpha > 60^\circ, \quad \beta, \gamma < 60^\circ$
In the first case both $A$ and $B$ could accommodate one vertex of $T_e$
In the second case only $A$ only could accommodate one vertex of $T_e$
In the first case (I), if the vertex chosen of $T_e$ is $A$ then, for the law of sines, the side $w$ of $T_e$ will be
$$\overline {AQ}  = w = c \cdot \frac{{\sin \beta }}{{\sin \left( {240^\circ  - \left( {\alpha  + \beta } \right)} \right)}}$$
on the other hand if the vertex chosen of $T_e$ is $B$ then it will be
$$\overline {BQ'}  = w = c \cdot \frac{{\sin \alpha }}{{\sin \left( {240^\circ  - \left( {\alpha  + \beta } \right)} \right)}}$$
Since $\sin \alpha>\sin \beta$ (this always holds with $ \alpha \ge \beta > 60^\circ$ and $\alpha+\beta\le 180^\circ$)
then the optimal strategy is to put the vertex of $T_e$ in $B$, that is in the vertex opposite the greatest angle.
In the second case (II) the only option will be to choose the vertex $A$ for one vertex of $T_e$ and it will be 
$$\overline {AQ}  = w = c \cdot \frac{{\sin \beta }}{{\sin \left( {240^\circ  - \left( {\alpha  + \beta } \right)} \right)}}$$
Why to put one vertex of $T_e$ on one vertex of  $T$
Moving the vertex of $T_e$ from, say $B$ to another point $B'$ its like working on another triangle $A', B', C$ with reduced size. The inner equilateral triangle will also have a reduced size (this part could be right but should probably need some more detail).
Conclusion
I think that the optimal strategy for positioning the $T_e$ with maximum size is:


*

*if there are two angles with amplitude $>60^\circ$ then put one
vertex of $T_e$ on the vertex opposite the one with greatest angle
aperture. A second vertex of $T_e$ will be on the side of $T$ opposite the vertex with the greatest angle aperture. These rules will determine the positions of the vertices of $T_e$ 

*if there is only one angle with amplitude $>60^\circ$ then put one
vertex of $T_e$ on this vertex as the other two won't fit for an
inscirbed equilateral triangle. A second vertex will be positioned on the
shortest side of the triangle chosen from the adjacent sides of the vertex with amplitude $>60^\circ$ 
A: Consider an arbitrary triangle PQR. Excluding the trivial case of an equilateral triangle, either one or two of the angles are at least 60 degrees, and wlog let P be the minimal example of these. I.e. P is the smallest angle of at least 60 degrees, and Q is either the largest angle or the second largest after P. Then the largest enclosed equilateral triangle (call it E) has one vertex at P.
If Q < 60 (i.e. P is the only angle >=60), then both other vertices are on QR.
If Q > 120 - P/2, then the second vertex lies on QR, at the intersection with a line drawn at an angle of 60 degrees from PR.
Otherwise the second vertex is at Q.
Motivation: The interesting case to consider is a triangle of angles 62, 89, 29 degrees. (Actually this is almost exactly the OP's triangle ABC above.

If Q<90, there will be a local maximum E based on P and Q. The diagram shows the perpendicular dropped from P to QR, and shows that there will be an equal E if it is possible to rotate this about P so that the vertex on QR is the same distance on the opposite side of the perpendicular. In this case, since angle Q is 89 degrees, we need to be able to rotate E through 2 degrees, which is exactly possible. Of course if angle Q is more than 90 degrees, rotation will always increase the size of E.
This is a sketch of an answer; it depends on proving that one vertex of E is at a vertex of PQR, and a messy set of cases for optimisation. But I hope I have captured the distinction between the two cases illustrated by (62, 89, 29). 
A: This is a late attempt to contribute to an interesting discussion I only recently became aware of. I hope the question is still alive.
If only one angle in the given triangle is greater than $60^o$, then the greatest inscribable equilateral triangle shares a vertex with this greatest angle and has its base on a portion of the opposite side, as in the OP's first figure. 
Presuming this as agreed on, and also that the greatest equilateral triangle must share at least one vertex with the given triangle, I consider only the case where two angles are greater than $60^o$, 
1) showing that the greatest equilateral triangle does not always share a vertex with the largest angle in the given triangle, while 
2) revealing the conditions under which it does, and 
3) the method for finding the greatest equilateral triangle when it does not. 
Suppose two angles of the given triangle are greater than $60^o$ and equal.

In isosceles triangle $ABD$, the greatest equilateral triangle has $AB$ as base when the base angles of $ABD$ are greater than $60^o$ but less than $80^o$, i.e.$$\frac{\pi}{3}<\angle DAB=\angle DBA<\frac{4\pi}{9}$$ For if on $AD$ we make $AE=AB$ and construct equilateral triangle $AEF$, vertex $F$ lies outside of triangle $ABD$ unless$$\angle ABF\le\angle ABD$$If $\angle ABF=\angle ABD$, as in the figure above, then $\triangle ABD$ and $\triangle BFA$ similar isosceles triangles, 
and since$$\angle BAF=180^o-2\angle ABF$$and also$$\angle ABF=\angle BAD=\angle BAF+60^o$$then$$180^o-2\angle ABF=\angle ABF-60^o$$making$$3\angle ABF=240^o$$and $$\angle ABF=80^o$$
Thus for a base angle of $80^o$, a greatest equilateral triangle can be constructed on a side of the isosceles triangle as well as on its base.
For base angles greater than $80^o$, however, i.e. when$$\angle ABF<\angle ABD$$ point $F$ lies within $\triangle ABD$, and the greatest equilateral triangle is not on the base of the isosceles triangle but on its side.

Accordingly, in the second figure (vertex $D$ of isosceles $\triangle ABD$ is out of view on the right), $F$ is within $\triangle ABD$. The maximum equilateral triangle $AHG$ is constructed by extending $AF$ to $H$ on $BD$, drawing a circle with radius $AH$, cutting $AD$ at $G$, and joining $HG$.
As $D$ becomes more and more remote, $AD$ and $BD$ approach parallelism, and $\triangle AHG$ approaches as a limit an equilateral triangle with height $AB$.  If $AB=1$, the side of this limiting greatest equilateral triangle is $\frac{2}{\sqrt 3}>1$.
If two angles of the given triangle are greater than $60^o$ but not equal, then the greatest equilateral triangle is either on the side between the vertices of these two angles, or shares a vertex only with the lesser of the two angles.
In the third figure, triangle $ABD$ has angles at $A$ and $B$ both greater than $60^o$, with $$\angle DAB>\angle DBA$$
If $\angle CBD<\angle ADB$, then point $F$ of equilateral triangle $AEF$ lies outside of $\triangle ABD$, so that the greatest equilateral triangle is either $\triangle ABC$ or lies on $BD$ with a vertex at $B$. 
But if, with $AD$ fixed in position, we move $D$ to the right, thereby increasing $\angle CBD$ and decreasing $\angle ADB$, until $\angle CBD=\angle ADB$, then point $F$ lies on $BD$ and $\triangle AEF$ is an equilateral triangle in $\triangle ABD$ equal to $\triangle ABC$.

Proof:
Since$$\angle EAC=60^o-\angle CAF$$and$$\angle CBD=\angle CBF=\frac{1}{2}\angle CAF$$then$$\angle EAC=60^o-2\angle CBD$$And since$$\angle ADB=180^o-2\cdot 60^o-\angle EAC-\angle CBD$$therefore$$\angle ADB=60^o-60^o+2\cdot \angle CBD-\angle CBD=\angle CBD$$
And so conversely, if $\angle CBD=\angle ADB$, then point $F$ lies on $BD$.
Now on $BD$ make $BG=AB$, and construct equilateral triangle $BGH$ equal to $\triangle ABC$. 
Then vertex $H$ lies within $\triangle ABD$. 
For suppose $H$ lies on $AD$: 
Then since $\angle ABH=\angle CBG$, by SAS we have$$\triangle ABH\cong \triangle CBG$$But$$\angle CBG=\angle ADB$$Therefore$$\triangle ABH\sim\triangle ADB$$But $\triangle ABH$ is isosceles. Therefore, $\triangle ADB$ is isosceles and$$\angle DAB=\angle DBA$$which is false. 
Therefore, $H$ lies within $\triangle ABD$, and, by the method already indicated in the second figure, an equilateral triangle greater than $\triangle BGH$ can be constructed on $BD$, sharing a vertex with the angle at $B$, which is less than the angle at $A$.
Although this case by itself, i.e. where $F$ lies on $BD$, proves that the conjecture is not always true, it appears that when two angles are greater than $60^o$ the conjecture is in fact universally false: when $D$ is remote enough so that $H$ and $F$ both lie within $\triangle ADB$, the greatest equilateral triangle inscribable on $BD$ with vertex at lesser angle $B$ is always greater than that inscribable on $AD$ with vertex at greater angle $A$.
In the daunting fourth figure, again$$\angle DAB>\angle DBA$$but $D$ is far enough out to the right in triangle $ADB$ so that $F$ and $H$ both lie within $\triangle ADB$. Extend $AF$ and $BH$ to cut $BD$ and $AD$ at $I$ and $J$, complete the larger equilateral triangles $ALI$ and $BJK$, and join $IJ$ and $KL$. 

Granting the apparent concyclicity of $J,A,B,I$ and $J,L,K,I$ (I need a proof), then since$$\angle JLK=\pi-\angle JIK$$then$$\angle JLK=\angle JIB$$But$$\angle JIB=\pi-\angle JAB$$Therefore$$\angle JLK=\pi-\angle JAB$$so that$$LK\parallel AB$$Hence $LK$ cuts the sides of $\triangle ADB$ proportionally, making$$\frac{AD}{DB}=\frac{AL}{BK}$$And since$$AD<BD$$therefore$$AL<BK$$and equilateral triangle $ALI$ sharing vertex $A$ of the greater angle $DAB$ is less than equilateral triangle $BJK$ sharing vertex $B$ of the lesser angle $DBA$.  
To conclude: apparently when the given triangle has two angles greater than $60^o$, if the greatest equilateral triangle shares a vertex with the greater of the two angles, it also shares a vertex with the lesser, but not vice-versa, i.e. the greatest equilateral triangle has as base either the side between the two angles greater than $60^o$, or a portion of the side opposite the greatest angle and shares a vertex with the lesser of the two angles greater than $60^o$.    
