Maximum area of triangle inside a convex polygon 
Prove that within any convex polygon of area $A$, there exists a triangle with area at least $cA$, where $c=\tfrac{3}{8}$. Are there any better constants $c$? 

I'm not sure how to approach this problem. It is easily proven that such a triangle should have its vertices on the perimeter of the polygon, but I don't know how to proceed from here. 
 A: There is not-so-painful way to prove a weaker inequality, i.e. that for any convex polygon $P$ with unit area there is a triangle $T\subset P$ with area $\geq \color{purple}{\frac{4}{21}}$. The approach is simple: let, counter-clockwise, $P_1,P_2,\ldots,$ $P_n=P_0,$ $ P_{n+1}=P_1$ be the vertices of the polygon, and let $P_{a-1}P_{a} P_{a+1}$ be the triangle with the smallest area among all the triangles of such a form. We remove the vertex $P_a$ and re-index the vertices, then continue the removal procedure until our polygon is a triangle. That is our $T$. It is not difficult to check that if the area of $P$ is $1$, the area of $T$ is at least
$$ \prod_{n\geq 4}\left(1-\frac{4\sin^2\frac{\pi}{n}}{n}\right)\geq \frac{4}{21}. $$
Some comments of mine to your revised question may be useful to improve the above inequality/approach: for instance, there is for sure a triangle $T\subset P$ with area
$$ \frac{3\sqrt{3}}{4\pi} \Delta(E^-) $$
where $E^-$ is the John-Loewner inellipse (the inscribed ellipse in $P$ with maximal area), and we have very accurate bounds when the vertices of $P$ all lie on a ellipse (so for any $n\leq 5$).
The original Blaschke argument is outlined in this answer by Christian Blatter.
Blaschke's approach is extremely slick: there is a Steiner symmetrization procedure that is area-preserving, so we may assume without loss of generality that the original polygon is a cyclic polygon. But that gives a way easier optimization problem, just related with the distribution of $n$ points on a circle.
A: As first pointed out by Mark Fischler in his comment, one can take $c = \frac{3\sqrt{3}}{4\pi}$.
In addition to Steiner symmetrization mentioned in Jack D'Aurizio's 
answer$\color{blue}{{}^{[1],[2]}}$,
there is another elegant analytic approach which can be generalized to inscribed $n$-gon.

E. Sas (1939) - For any $n \ge 3$, let $c_n = \frac{n}{2\pi}\sin\left(\frac{2\pi}{n}\right)$. For any convex body $B$ in the plane, there exists a $n$-gon $P$ inside $B$ such that $$\verb/Area/(P) \ge c_n \verb/Area/(B) 
\tag{*1}$$

When $n = 3$ and $B$ is a convex polygon, the claim we can take $c = c_3 = \frac{3\sqrt{3}}{4\pi}$ follows immediately. 
Following argument is based on a paper by E. Sas in German$\color{blue}{{}^{[3]}}$. All mistakes are mine and in case anything looks fishy. Please refer to E. Sas paper for correct statement.

Since I am lazy, I will assume 
$B$ is a convex body whose boundary $\partial B$ is a smooth Jordan curve.
This avoids all sort of potential pathologies and save me from justifying a lot of stuff.
Let $2\ell$ be diameter of $B$. Let $L$, $R$ be two points on $\partial B$ at a  distance $2\ell$ apart. Choose a coordinate system such that $L,R$ is located at $(-\ell,0)$ and $(\ell,0)$ respectively. Under such a coordinate system, $\partial B$ has a parametrization $\gamma$ of the form:
$$[0,2\pi] \ni t \quad\mapsto\quad \gamma(t) = (\ell\cos t, e(t)\sin t ) \in \partial B$$
where $e(t) > 0$ is some smooth function. Extend $\gamma(t)$ and $e(t)$ to smooth periodic functions over $\mathbb{R}$ with period $2\pi$.
For any fixed $t \in [0,2\pi]$ and $k \in \mathbb{Z}$, let $t_k = t + \frac{2k\pi}{n}$. It is clear $t_{k+n} = t_k$. 
Let $P(t)$ be the $n$-gon with vertices $\gamma(t_0), \gamma(t_1), \gamma(t_2), \ldots, \gamma(t_{n-1})$. 
Since $B$ is convex and $\gamma(t_k) \in B$, $P$ lies inside $B$.
Let $f(t)$ be the area of $P(t)$. It is easy to work out
$$f(t) = \frac{\ell}{2} \sum_{k=1}^n e(t_k)\sin t_k(\cos(t_{k-1}) - \cos(t_{k+1})) = \ell\sin\left(\frac{2\pi}{n}\right)\sum_{k=1}^n e(t_k)\sin^2(t_k)$$
Now treat $t$ as a variable and average $f(t)$ over $[0,2\pi]$, one get
$$\frac{1}{2\pi}\int_0^{2\pi} f(t) dt =
\frac{n}{2\pi}\sin\left(\frac{2\pi}{n}\right)\times
\ell\int_0^{2\pi} e(t)\sin^2(t)dt = c_n \verb/Area/(B)$$
This implies there exists a $t_{*} \in [0,2\pi]$ such that $f(t_{*}) \ge c_n \verb/Area/(B)$. In other words,
there exists a poylgon $P = P(t_{*}) \subset B$ whose area is at least $c_n$ of that of $B$.
Back to the problem at hand for convex polygon. 
For any polygon $P$, let $|P|$ be its number of sides.
Given any convex polygon $Q$ and any $0 < c < c_n$, approximate $Q$ by a convex body $B$ with smooth boundary whose area is at least $\frac{c}{c_n}\verb/Area/(Q)$. By $(*1)$, there is a $n$-gon $P \subset B \subset Q$ with 
$\verb/Area/(P) \ge c_n\verb/Area/(B) \ge c\verb/Area/(Q)$. Since the set of polygon $P \subset Q$ with $|P| \le n$ is compact under the topology induced from $\mathbb{R}^2$ and $\verb/Area/(\cdot)$ is continuous with respect to this topology, there is a polygon $P_{*} \subset Q$ with $|P_{*}| \le n$ and $\verb/Area/(P_*) \ge c_n \verb/Area/(Q)$. If $|P_{*}| < n$, we can turn $P_*$ to a $n$-gon by adding some extra vertices along its edges.
Fix $n$ to $3$, this means there is a triangle inside $Q$ whose area at least $c_3 = \frac{3\sqrt{3}}{4\pi}$ of that of $Q$.
Notes


*

*$\color{blue}{[1]}$ - For a proof of $(*1)$ when $n = 3$, see this answer by Christian Blatter. 

*$\color{blue}{[2]}$ - Christian Blatter's answer uses Steiner symmetrization. For more detail, please refer to
Wilhelm Blaschke, Über affine Geometrie III: Eine Minimumeigenschaft der Ellipse. Leipziger Berichte 69 (1917), pages 3–12. 

*$\color{blue}{[3]}$ - E. Sas, Über eine Extremaleigenschaft der Ellipsen, Compositio Math. 6 (1939), 468–470.
A: The statement is true using any $c < \frac{3\sqrt{3}}{4\pi} \approx 0.413$, which is more than $ \frac38$. But I have tried and failed to find an easy proof.
EDITED A DAY LATER
If you impose a restriction on the number $s$ of sides in the convex polygon, the $c$ for that set of polygons becomes greater.  For  $s=3$, $c=1$ of course.  For $s=4$ it is easy to show that $c=\frac12$: $c$ is at least $\frac12$, as demonstrated by a line connecting two non-adjacent vertices (one of the triangles thus formed must have at least half the area); and  $c$ is at least $\frac12$, as demonstrated by a square.
A: Let's turn the problem on its head.  Let triangle $\Delta abc$ be a maximum area triangle contained in convex polygon $\mathscr{P}$.  We are asked to show that the area of the triangle is at least $C$ times the area $A$ of $\mathscr{P}$, where $C=3/8$.  We offer a simple proof that $C \ge 1/4$ can be achieved (and improved upon).
As noted in the Question, the points $a,b,c$ must lie on the perimeter of $\mathscr{P}$. Further these points may be chosen, without loss of generality, to be vertices of the polygon $\mathscr{P}$.  To wit: If (say) vertex $a$ is in the interior of an edge $e$ of the polygon, then sliding $a$ one way or the other would increase the area of $\triangle abc$ unless the edge $e$ is parallel to the side $bc$ of the triangle opposite to $a$ (by the familiar formula for area of a triangle being one-half the height of the triangle with respect to base $bc$).
Now any affine transformation applied to $\mathscr{P}$ preserves the ratio of its area to that of a triangle inscribed within it.  Therefore we may apply some affine transformation that maps $\triangle abc$ to an equilateral triangle, and the ratio of areas remains the same.  It also preserves convexity of $\mathscr{P}$, and the identification of $a,b,c$ with vertices of $\mathscr{P}$. Henceforth we will simply refer to the transformed (equilateral) triangle as $\triangle abc$ and the transformed polygon as $\mathscr{P}$.
A picture may help the Reader to follow our argument to its next conclusion, as we draw lines through the vertices of an equilateral triangle parallel to their respective opposing sides:

Fig. 1 Equilateral triangle $\triangle abc$ with lines through vertices
Here $\triangle abc$ is the central equilateral triangle.  No point $p$ of $\mathscr{P}$ can lie outside the circumscribing equilateral triangle, because to do so would induce a triangle with area strictly greater than $\triangle abc$.  For if (say) point $p$ were on the outside of the outer line through $a$, opposite from side $bc$, then $\triangle pbc$ would have area strictly greater than $\triangle abc$.
Therefore $\mathscr{P}$ is constrained to have area no greater than four times the area of $\triangle abc$.  This proves that $C \ge 1/4$ can be achieved, that the maximum area of an inscribed triangle is at least one-fourth the area of convex polygon $\mathscr{P}$.
I know a somewhat tedious but elementary way to improve this to $C = 3/11$, still a good ways short of the desired $C = 3/8$, but suggesting that improvements on this argument might be worth pursuing.
