What is the maximum number of triangles that can be formed by joining the vertices of a decagon? I think we have to assume that diagonals are line segments.
I know that the number of triangles that have $3$ vertices such that those 3 are the vertices of the decagon is $\binom{10}3$ but after this it just gets crazy, There are $35$ diagonals and $10$ line segments how do we count the triangles that get formed inside the decagon and how do we maximize it?
 A: You seem uncertain about whether the diagonals are lines or line segments, and I suppose the same would go for the sides. Since the conditions of the question are not exactly settled, let me suggest no holds barred. Let all lines be produced indefinitely. Also, contrary to an earlier response, I see no reason why every triangle vertex must also be a vertex of the decagon.
Here is an image to illustrate my meaning. The decagon is in black. Each of the three blue lines is formed by connecting vertices of the decagon. They define the red triangle, none of whose vertices happen to be vertices of the decagon.

Any two of the vertices define a line.
number of lines $= \binom{10}2 = 45$
Having defined these 45 lines, the greatest number of triangles would result if no two of the lines are parallel or coincident, and no three of the lines are concurrent. That certainly is a possibility. In that case, any three of the lines define a triangle.
maximum number of triangles $= \binom{45}3 = 14,190$
A: Consider how to choose the vertices.  There are ten points you could choose for the "first" vertex, the nine points left for the "second vertex", and 8 for the "third".  That gives 10(9)(8)= 720 choices.
But, of course, there are no "first","second", or "third" vertices in a triangle. We do not want to count the same three vertices, in a different order, as a different triangle.  There are 3!= 6 different orders for the same three vertices.  720 has counted each triangle 6 times so the correct number is 720/6= 120 triangles.
