Maximizing the probability of multivariate hypergeometric distribution Suppose there are $R$ number of red balls, $B$ number of black balls, and $W$ number of white balls in an urn. The total number of balls are $N$, thus, $N = R + B + W$. I draw $M$ number of balls from the urn, where $N \geq M \geq B$. The probability of drawing all the black balls, but none of the white balls follows the multivariate hypergeometric distribution:
$$P = \frac{\binom{N-B-W}{M-B} \cdot \binom{B}{B}}{\binom{N}{M}}$$.
Now I want to find the value of $M$ that maximizes the probability $P$. I was initially thinking of taking the derivative of $P$ with respect to $M$ by transforming the binomial coefficients into gamma functions, but it was too complicated. Any suggestion?
 A: Since you MUST choose all black balls, we can ignore the second factor in the numerator. Now, let $\delta = M-B$, we can then re-express the probability as:
$P=\frac{R \choose \delta}{N \choose B+\delta}=\frac{R!(B+\delta)!(W+R-\delta)!}{\delta!(R-\delta)!N!}$.
When $\delta=0$ we get $P_0 = \frac{B!(W+R)!}{N!}$. If we increase to $\delta=1$, the numerator will change by a factor of $\frac{B+1}{W+R}$ and the denominator will change by a factor of $\frac{1}{R}$. Putting this together, we get a total change factor of: $\frac{R(B+1)}{W+R}$. Increasing $\delta$ by 1 to get $\delta=2$, we will further change the probability by a factor of $\frac{(R-1)(B+2)}{2(W+R-1)}$.
As long as each of these factors are $\geq1$, we should increase $\delta$. Therefore, we want to find $\max\delta: \frac{(R-\delta+1)(B+\delta)}{\max\{1,\delta\}(W+R-\delta+1)}\geq1$. If we solve the continuous relaxation for the case where LHS=1, we get the equation $(R-\delta+1)(B+\delta)=\delta (W+R-\delta+1)$ where $\delta \in [1,R]$
Expanding the terms, we get: $RB+R\delta-\delta B-\delta^2+B+\delta = W\delta+R\delta-\delta^2+\delta$.
Cancelling like terms on both sides, we get the simplified equation:
$RB-\delta B+B = W\delta$, where we get $\delta = \lfloor \frac{B(R+1)}{W+B}\rfloor$ as the optimal value. 
As an example, if $W=5, B=7, \text{and}\; R=4$, we get $\delta = \lfloor \frac{7\times 5}{5+7} \rfloor = \lfloor \frac{35}{12}\rfloor = 2 $ so you would choose $M=B+2 = 9$ balls.
