I had learned probability long ago and am trying to re-learn it.

I came across an exercise while reading about forward probabilities: An urn contains K balls: B black balls and K - B white balls. A ball is randomly drawn and replaced from the urn, N times. What is the probability distribution of the number of times a black ball is drawn, $n_{B}$?

I already have the answer to this question, I just don't understand how it is calculated. The answer is: let $f_{B} \equiv \frac{B}{K}$ then $P(n_{B}|f_{B}, N) = \left( \begin{array}{c} N \\ n_{B} \end{array} \right) f^{n_{B}}_{B}(1 - f_{B})^{N - n_{B}}$

I tried to search in this website and couldn't find the answer, that's the reason I'm creating a new question for it. If you know there's an answer somewhere, just point me to it. Thank you very much.


PS: if you're curious about where I saw this exercise, it's in Information Theory, Inference and Learning Algorithms, exercise 2.4 page 27, its answer in page 40.


I will change your notation a little: complicated notation can get in the way of understanding. On a draw, the probability of drawing a black ball is $\frac{B}{K}$. Call this number $p$.

Call drawing a black ball a "success." On any draw, the probability of success is $p$, and therefore the probability of failure (white ball) is $1-p$.

Repeat the experiment independently $n$ times. We will find an expression for the probability that the number of successes is exactly $k$.

The number of successes has Binomial Distribution (look at the Wikipedia for a lot more detail than we will have below.)

It is convenient for the exposition to choose explicit numerical values for $n$ and $k$. Pick for example $n=8$ and $k=3$. So we want the probability of $3$ successes in $8$ trials.

Record what happened on your trials as a string made up of the letters B and W. So for example BBWBWWWW records an experiment where the first two balls drawn were black, the third white, the fourth black, and the rest white.

The probability of getting this particular string is $(p)(p)(1-p)(p)(1-p)(1-p)(1-p)(1-p)$, or more simply $p^3(1-p)^5$.

But there are other ways we could have gotten $3$ black, like WWBBWBWW. The probability of this is $(1-p)(1-p)(p)(p)(1-p)(p)(1-p)(1-p)$, again $p^3(1-p)^5$.

Now let us count the number of ways to get $3$ black. We need to nmake an $8$-letter "word" that has exactly $3$ B and $5$ W. The locations of the B can be chosen in $\binom{8}{3}$ ways. Once this is done, the word is determined. So there are $\binom{8}{3}$ patterns that yield $3$ black. Each pattern has probability $p^3(1-p)^5$, so the probability of $3$ black is $$\binom{8}{3}p^3(1-p)^5.$$

Exactly the same reasoning shows that if we repeat the experiment $n$ times, the probability of $k$ successes is $$\binom{n}{k}p^k(1-p)^{n-k}.$$

  • $\begingroup$ Your answer and example are very clear. Thanks a lot :) $\endgroup$ – Long Thai Sep 25 '12 at 20:56

If you know about the binomial distribution, this is just an application of it. Otherwise you can see it this way: Let $$ \square \square \square\ldots \square $$ be your $N$ balls. Now choose the place where you want to draw the black ones. This can be done in ${N \choose n_B}$ ways. The probability of exactly $n_B$ black balls and $N-n_B$ white balls is $f_{B}^{N_B}$ for exactly $n_B$ black ones times $(1-f_B)^{N-n_B}$ for exactly $N-n_B$ white balls. Since we said there were ${N \choose n_B}$ ways to order them, you get the desired result.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.