(Note: I'm not sure this is the right answer since it disagrees with OP's calculation, so I'd appreciate it if someone would check the solution.)
The letters in the word MONSTER are all distinct, so the question is how many ways we can form a string of $6$ characters with each character chosen from these $7$ choices, under the restriction that there are three different letters. To count this, we can choose which three letters they are first: a factor of $\binom73$. Once we've chosen the letters, we need to look at how to arrange them.
We can do so by cases: call the three letters $A,B,C$, then either there are $4$ of one letter and $1$ of the other two, or $3$ of one letter and $2$ and $1$ respectively of the other two, or $2$ of each letter. In the first case there are $\binom{6}{4}\binom{3}{1}\binom{2}{1}$ ways, in the second there are $\binom{6}{3}\binom{3}{1}\binom{3}{1}\binom{2}{1}$ ways, and in the third there are $\binom{6}{2}\binom{4}{2}\binom{3}{1}\binom{2}{1}$ ways. So the answer is
$$\binom73\left[\binom{6}{4}\binom{3}{1}\binom{2}{1}+\binom{6}{3}\binom{3}{1}\binom{3}{1}\binom{2}{1}+\binom{6}{2}\binom{4}{2}\binom{3}{1}\binom{2}{1}\right]=34650.$$
Edit: $18900$ is correct, as explained in Matthew Daly's answer. I think I've figured out where I went wrong above: in counting the third case, the quantity $\binom62\binom42$ was intended to be the number of ways to partition a set of $6$ items into three $2$-member subsets, the logic being that we first choose one such subset and then choose another in the remaining $4$ items. That's wrong however, as it counts each possibility $3!=6$ times (according to the order in which the partitions are chosen). So the correct answer should actually be
$$\binom73\left[\binom{6}{4}\binom{3}{1}\binom{2}{1}+\binom{6}{3}\binom{3}{1}\binom{3}{1}\binom{2}{1}+\frac1{3!}\binom{6}{2}\binom{4}{2}\binom{3}{1}\binom{2}{1}\right]=18900.$$