How many ways are there of coloring $n$ numbers (using $k$ colors) s.t. each color is used at most $d$ times? Let's assume we have $n$ numbered items and $k$ colors. We color each of the items with a single color. How many such colorings exist such that each of the colors is used at most $d$ times?
 A: Here's a simple case; as I mentioned, general case involves partition of sets and probably doesn't have a simple form. 
Consider $n=3, k=20, d=1$. Then $a_k \in \{0,1 \}$ is the number of times you use k-th colour. You want $\sum_{k=1}^{20} a_k = 3$, or in other words $1+1+ \ldots 1 +\ldots 0 =3$ , so you need to select 3 colours out of 20, but, since all the items are distinct, colouring 1,2,3 and 3,2,1 are different, so you need to multiply by $3!$. Hence, the result is 
$$
\binom{20}{3} \cdot 3!
$$ 
EDIT: for $d=3$, since we have $n=3$ we should consider 3 cases: the one above, 2 balls of the same color and 1 different, and all 3 balls of the same colour. The last case is trivial, it's $\binom{20}{1}$ obviously. For the 2-1 case we need to consider all possibilities of selecting 2 out 20 and 1 out remaining 18 and reallocating them (e.g. BBW, BWB, WBB): this is $\binom{20}{2} \binom{18}{1} \cdot 3$.
A: Here's a solution using exponential generating functions. Let's start with the base case of just one color. The number of ways to color $i$ numbered items with one color is just 1, as long as the number of items does not exceed $d$; otherwise, there are no ways to color the $i$ items. The exponential generating function for this sequence is $\sum_{i=0}^d\frac{x^i}{i!}$. Now, let's generalize to $k$ colors. If there are $a_i$ ways to color $i$ numbered items with one color, then the number of ways to color $n$ numbered items with $k$ colors is $\sum_{i_1,\dots,i_k}\binom n{i_1,\dots,i_k}a_{i_1}\dots a_{i_k}$. Therefore, the exponential generating function of the number of ways to color numbered items with $k$ colors with at most $d$ of each color is just the power $\left(\sum_{i=0}^d\frac{x^i}{i!}\right)^k$. In other words, the answer to your question is just the coefficient $$\left[\frac{x^n}{n!}\right]\left(\sum_{i=0}^d\frac{x^i}{i!}\right)^k.$$
To extract this coefficient, we can use the multinomial theorem $$\left(\sum_{i=0}^d\frac{x^i}{i!}\right)^k=\sum_{a_0+\dots+a_d=k}\binom k{a_0\dots a_d}\prod_{i=0}^d\left(\frac{x^i}{i!}\right)^{a_i}.$$ Then, the coefficient of $x^n/n!$ in this expression, that is the number of ways to color $n$ numbered items with $k$ colors using each color at most $d$ times, is 
$$n!\sum\binom k{a_0\dots a_d}\prod_{i=0}^d(i!)^{-a_i},$$
where the sum ranges over all tuples $a_0,\dots,a_d$ of non-negative integers such that $a_0+a_1+\dots+a_d=k$ and $0 a_0+1a_1+\dots+da_d=n$. Granted, this index set is messy (so Alex's comment about not having a "nice solution" is probably true). Nevertheless, it's an explicit answer.
