How to prove this geometric inequality and find the equality cases? 
In triangle $ABC$, denote by $a,b,c,$ the side lengths, and $F$ the area.
Prove that : $F \leq {\frac {1}{16}}(3a^2+2b^2+2c^2)$
and determine when the equality holds. Can we find another set of coefficients of $a^2,b^2,$ and $c^2$ for which equality holds ?

 A: One way to compute the area of a triangle from its edge lengths would be Heron's formula:
$$F=\frac14\sqrt{(a^2+b^2+c^2)^2-2(a^4+b^4+c^4)}$$
So one thing we can do is square both your formula and this one. As we know both formulas will result in a positive number, squaring does not change the inequality. After putting Heron's formula into your inequality and squaring both sides we obtain
\begin{align*}
\frac1{16}\left((a^2+b^2+c^2)^2-2(a^4+b^4+c^4)\right)
&\le\frac1{16^2}\left(3a^2+2b^2+2c^2\right)^2 \\
16\left((a^2+b^2+c^2)^2-2(a^4+b^4+c^4)\right)
&\le\left(3a^2+2b^2+2c^2\right)^2 \\
20a^2b^2+20a^2c^2+24b^2c^2 &\le 25a^4+20b^4+20c^4 \\
4b^2c^2-5a^2&\le 10\left((a^2-b^2)^2+(a^2-c^2)^2+(b^2-c^2)^2\right)
\end{align*}
From the first row to the second I just got rid of the fractions. From the second to the third I took the difference and expanded it using my computer algebra software, then put negative terms back on the other side. In the last row I attempted to look for some structure, concentrating on squares as we know these are positive.
So what can we learn? If $a=b=c$ the right side will be zero, but the left will be negative and the inequality holds. To obtain an example of a situation where the equality holds, we could assume $a=1,b=c$. Then the equation becomes $4b^4-5=20(1-b^2)^2$. This equation has one positive solution, namely $b=\frac12\sqrt5\approx1.12$, which leads to $F=\frac12=\frac{1}{16}(3a^2+2b^2+2c^2)$. So it is possible to achieve equality. Let's hunt for the set of all such situations. Notice how
$$20a^2b^2+20a^2c^2+24b^2c^2 = 25a^4+20b^4+20c^4$$
is a homogeneous quadratic equation in squared distances. You can think of this as a description of a conic section in homogeneous coordinates. You can also write it in matrix notation as
$$(a^2,b^2,c^2)\cdot\begin{pmatrix}
-25 & 10 & 10 \\
10 & -20 & 12 \\
10 & 12 & -20
\end{pmatrix}\cdot\begin{pmatrix}a^2\\b^2\\c^2\end{pmatrix}=0$$
But the determinant of this matrix is zero, so it is a degenerate conic, which consists of a pair of lines. Put differently, the matrix has rank two, and can therefore be written as the symmetric product of two vectors. You can write that matrix as $-\frac12(g\cdot h^T+h\cdot g^T)$ with
$$g=\begin{pmatrix}-5\\2-4i\\2+4i\end{pmatrix}\qquad
h=\begin{pmatrix}-5\\2+4i\\2-4i\end{pmatrix}$$
so there are two elementary conditions, either of which will make your equality true. They can be written in a single line as they are complex conjugates of one another:
$$-5a^2+(2\mp4i)b^2+(2\pm4i)c^2=0$$
The only real solution is using $b^2=c^2$ so that the imaginary components there cancel out. So the example I obtained above already covers the essential situation, except for scale. Picking any positive $a$ you can use $b=c=\frac12\sqrt5a$ to achieve equality. If you want to consider degenerate triangles as well, you might also use $a=b=c=0$ of course.
As there is only a single line in your 3d parameter space where the equation holds with equality, you can connect any point not on that line to any other point not on that line using a continuous path that doesn't cross the line. Since a continuous change in parameters can't make the resulting difference (or inequality) go from positive to negative (resp. from $<$ to $>$) without going through zero (resp. equality), this means that the inequality you posted has to hold for all real choices of these three parameters.
If you want a different set of coefficients, observe how the fact that the determinant of the matrix above being zero was a neccessary (although not sufficient) condition for there being exactly one solution. You could do the computation using variables for your coefficients and then obtain a condition on these variables describing situations where there will be exactly one solution (up to scale) where the equality holds. Make sure the two factors of the matrix are complex, not real.
If you need coefficients for which the inequality holds strictly (i.e. equality isn't possible), you might find a non-degenerate matrix (i.e. non-zero determinant) which describes a purely complex conic. That's the case if all the eigenvalues of the matrix have the same sign. You can check for the single-real-solution scenario by looking at the eigenvalues, too: you'd have one zero and two eigenvalues of equal sign.
