I was taking Caltech - ML Course and solving problem 1.3 in this link

We have 2 opaque bags, each containing 2 balls. One bag has 2 black balls and the other has a black ball and a white ball. You pick a bag at random and then pick one of the balls in that bag at random. When you look at the ball, it is black. You now pick the second ball from that same bag. What is the probability that this ball is also black?

I tried to do this way:

  1. Since we are trying to find the probability of 2nd being ball being black when we already know that first ball is black. This reduces our problem to probability by which we picked the bag with two black balls.

So the answer should be 1/2.

PS: This is not a homework/assignment. This course has been already finished. I am just taking it offline for learning purpose.

  • $\begingroup$ You happened to forget that the bags contain a white ball, a black ball, but another black ball as well AFTER the first removal. $\endgroup$ – Don Larynx Sep 23 '13 at 0:05

Since $2$ of the three black balls are in the $2$-blacks bag, the required probability is $2/3$.

Since intuition can be treacherous, we will in addition do a formal conditional probability calculation.

Let $F$ be the event the first ball is black. The probability of $F$ is $3/4$. For with probability $1/2$, we chose the $2$-blacks bag, in which case the probability of black is $1$, and with probability $1/2$ we chose the mixed bag, in which case the probability of black is $1/2$. Thus $\Pr(F)=(1/2)(1)+(1/2)(1/2)$.

Let $S$ be the event the second ball drawn is black. The probability of $S$ is $1/2$, for $S$ happens precisely if we picked from the $2$-blacks bag.

The event $F\cap S$ happens precisely if $S$ happens. Thus $$\Pr(S|F)=\Pr(F\cap S)/\Pr(F)=\frac{2}{3}.$$

  • $\begingroup$ Thanks @Andre..This was really helpful $\endgroup$ – sachinjain024 Sep 23 '13 at 5:39
  • $\begingroup$ You are welcome. In conditional probability problems, there are potential pitfalls, so you were right in wanting a formal calculation. But informally, getting a black on the first try makes it more likely that we drew from the $2$-blacks bag. $\endgroup$ – André Nicolas Sep 23 '13 at 6:00

There is a clear mistake in Blunderboy's explanation.

His answer uses the Bayes formula but then the term P(E) is erroneously excluded from the computation. In addition the value for P(A|B) is simply wrong.

This is the correct formulation of the problem using Bayes theorem:

P(A|E) = 1 ( if we choose the bag with two black balls the probability that we select a black ball is 100%)
P(E) = 1/2 ( each bag can be chosen with equal probability)
P(A)= 3/4 as explained by Andre Nicolas.

Then the Bayes theorem leads us to:

P(E|A)= P(A|E).P(E)/P(A) = (1 * 1/2)/(3/4)=2/3.

Andre Nicolas answer, which uses the Kolmogorov definition instead of the Bayes theorem, is completely correct.


In addition to @blunderboy's explanation I found this New York Times article very helpful in understanding Bayes’s theorem. Also checkout wikipedia, which shows the solution of the example mentioned in the NYT.

Here is the original quote from the NYT together with the example:

The theorem itself can be stated simply. Beginning with a provisional hypothesis about the world (there are, of course, no other kinds), we assign to it an initial probability called the prior probability or simply the prior. After actively collecting or happening upon some potentially relevant evidence, we use Bayes’s theorem to recalculate the probability of the hypothesis in light of the new evidence. This revised probability is called the posterior probability or simply the posterior. Specifically Bayes’s theorem states (trumpets sound here) that the posterior probability of a hypothesis is equal to the product of (a) the prior probability of the hypothesis and (b) the conditional probability of the evidence given the hypothesis, divided by (c) the probability of the new evidence.

Consider a concrete example. Assume that you’re presented with three coins, two of them fair and the other a counterfeit that always lands heads. If you randomly pick one of the three coins, the probability that it’s the counterfeit is 1 in 3. This is the prior probability of the hypothesis that the coin is counterfeit. Now after picking the coin, you flip it three times and observe that it lands heads each time. Seeing this new evidence that your chosen coin has landed heads three times in a row, you want to know the revised posterior probability that it is the counterfeit. The answer to this question, found using Bayes’s theorem (calculation mercifully omitted), is 4 in 5. You thus revise your probability estimate of the coin’s being counterfeit upward from 1 in 3 to 4 in 5.

  • $\begingroup$ Welcome to math.stackexchange.com and thanks for sharing the link. On a side note, link-only answers are discouraged here. Instead put them as comment. $\endgroup$ – sachinjain024 Mar 3 '14 at 14:13

Here is how we can proceed. Take this approach. Since first ball drawn is black and you want to find out the probability of second ball being drawn is black So it is the same as probability of telling the bag chosen was the one with 2 black balls.

Hence, use Bayes's theorem and consider these events

A = First ball drawn is black
E = Bag chosen was one with 2B balls
P(A|E) = 1/2
P(A) = 1/2*1 + 1/2*1/2 = 3/4
P(E|A) = P(A|E).P(E)/P(A)
       = (1/2)/(3/4)
       = 2/3 which is correct answer.

I am taking the same course, however the question hinted that the solution could be found using Bayes' Theorem.

Although intuitively the answer is $\frac{2}{3}$, using Bayes' Rule actually made finding the solution more difficult. Reformulating the question in the context of Bayes', one can ask "what is the probability after selecting one black marble, that the black marble came from the bag with two black marbles."

Clearly, the probability of selecting two black marbles in a row in the same as selecting one black marble from the bag with two black marbles! And this is where Bay's rules comes into play.

$P = \frac{(0.5)(\frac{2}{2})}{(0.5)(\frac{2}{2}) + (0.5)(\frac{1}{2})}$

$P = \frac{\frac{1}{2}}{\frac{1}{2} + \frac{1}{4}}$

$P = \frac{\frac{1}{2}}{\frac{2}{4} + \frac{1}{4}}$

$P = \frac{\frac{1}{2}}{\frac{3}{4}}$

$P = \frac{1}{2}\cdot \frac{4}{3}$

$P = \frac{4}{6}$

$P = \frac{2}{3}$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.