Probability of drawing 3 red balls first 
You have 3 red, 3 green, and 3 blue balls. You draw 1 at a time. What
  is the probability of picking all red balls before all other balls?

The probability of drawing 3 red ball at once (I thought) is $(\frac39)(\frac28)(\frac17)$ 
Clearly this is not right, because the answer is: $9\choose3$$(3!)^3$ $/9!$
The only thing I understand about the real answer is that $9!$ is the sample space.
How was this answer figured out?
 A: Assuming the interpretation of the question is to find the probability of picking all red balls before any other ball.  E.g. RRRBGBBGG is allowed, but RBRGGGRBB is not.
Your answer is correct and can be seen easily via a tree diagram and the multiplication principle of probability.  Let $R_1$ denote the event that a red ball was pulled in round one, $R_2$ denote the event that a red was pulled in round two, etc... The problem then asks us to find $Pr(R_1\cap R_2\cap R_3)$
$$Pr(R_1\cap R_2\cap R_3) = Pr(R_1)\cdot Pr(R_2\mid R_1)\cdot Pr(R_3\mid R_1\cap R_2) = \frac{3}{9}\cdot \frac{2}{8}\cdot \frac{1}{7}$$
What is likely meant by the alternative solution is instead:
$$\frac{\binom{\color{red}{6}}{3}(3!)^3}{9!}$$
There is either a typo in your post or in the answer key.
Let us temporarily assume that the balls are numbered.  Red1, Red2, ...
Let our sample space be all ways in which we can order the nine balls.  The size of the sample space is then $9!$.
Let us count how many orderings of the balls satisfy the condition that the three red balls are in the front via multiplication principle.


*

*Choose the positions occupied by the red balls.  $1$ choice (they must occupy the first three spaces)

*Choose the positions occupied by the blue balls.  $\binom{6}{3}$ choices (they will occupy three of the remaining six positions)

*From left to right, in the positions reserved for red balls, decide which red ball is where.  $3!$ choices ($3$ for the first position, $2$ remaining for the second position, $1$ remaining for the final position)

*Similarly, choose the order for the blue balls.  $3!$ choices

*Similarly, choose the order for the green balls.  $3!$ choices


Putting all of this together, we have the final answer $$\frac{\binom{6}{3}(3!)^3}{9!} = \frac{1}{84}$$
Your answer agrees with this:
$$\frac{3}{9}\cdot\frac{2}{8}\cdot\frac{1}{7}=\frac{1}{84}$$

In the event that the question is instead asking us to find the probability of picking all red balls before the last non-red ball is picked (RBRGGGRBB is one of the allowed outcomes), we may approach directly:


*

*Pick the locations of the red balls.  $\binom{8}{3}$ choices (The last position is not allowed, but all eight others are)

*Pick the locations of the blue balls.  $\binom{6}{3}$ choices (we want to choose three out of the six remaining positions)

*Pick the order of the red balls

*Pick the order of the blue balls

*Pick the order of the green balls


Giving a probability of $\frac{\binom{8}{3}\binom{6}{3}(3!)^3}{9!}=\frac{2}{3}$
This could have been answered more easily by noting that we could draw the balls in reverse order (draw the last ball at the beginning, on to the first ball at the end).  The probability then of this occurring is $1-Pr(R_1)=1-\frac{3}{9}=\frac{2}{3}$
