Difference between Conditional Probability and Bayes Theorem

Question

To ask my question I will present an example. Suppose we have $2$ boxes with yellow and blue balls. Box 1 has $5$ yellow and $4$ blue balls and Box 2 has $7$ yellow and $5$ blue balls. We draw a ball at random from one of the boxes ($50-50$ chance) and the ball is yellow. What is the probability of the ball originating from Box 1?

To solve this, I used conditional probability which I understand is wrong and the correct answer would be to use Bayes theorem, which I didn't even know about. However because the chance is $50-50$ for either box the results were very similar.

My question is, what is the difference between the two formulas?

Answer 1

First I explain why there is no real difference between both, then I tackle your example.

There is no fundamental difference between conditional probability and Bayes' theorem. Actually, Bayes' theorem is a simple rewrite of the definition of conditional probability.

Let $A$ and $B$ be two events.

The conditional probability of event $A$ knowing $B$ occurred is:

$$P(A \mid B) = \frac{P(A \cap B)}{P(B)}$$

Basically this formula restricts the sample space to what happens within event B, and normalize the new probability measure by dividing by $P(B)$ to ensure the sum is $1$ as it should be.

If you manipulate this expression, you find:

$$P(A \cap B) = P(A\mid B)\,P(B)$$

Of course, by symmetry, replace $A$ by $B$ and you get:

$$P(B \cap A) = P(B\mid A)\,P(A)$$

Now, Bayes' theorem simply merges these two expressions in a different form. In a way, it's not really a "theorem" because its proof is trivial. But its implication for statistics are so wide that it's called "theorem"

Bayes theorem:

$$P(A \mid B) = \frac{P(B \cap A)}{P(B)} = \frac{P(B \mid A)\,P(A)}{P(B)}$$

Asking the difference between Bayes' theorem and conditional probability is like asking the difference between these two equations: $$ x = \frac{a}{b} \quad \text{ and } \quad b\times x = a $$

Hope this helps

---

Edit:

to tackle your example:

Let $B_1$ the event "the ball is drawn from box 1" and $B_2$ the event "the ball is drawn from box 2". We know that $P(B_1) = P(B_2) = 0.5$.

In box 1, there are 5 yellow and 4 blue balls so if $Y$ is the event "we draw a yellow ball", then $P(Y \mid B_1) = 5 / (4+5) = 5/9$.

Likewise for box 2, $P(Y \mid B_2) = 7 / (7 + 5) = 7 / 12$

With this setup, we have:

$$P(Y) = P(Y \mid B_1)\,P(B_1) + P(Y \mid B_2)\,P(B_2)$$ $$P(Y) = 5/9 * 0.5 + 7/12 * 0.5 \approx 0.56944444$$

This formula I used is called "law of total probability". It can be used when two (or more) events partition the sample space $\Omega$. That is: $B_1 \cap B_2 = \emptyset$ and $B_1 \cup B_2 = \Omega$.

Now, you can compute $P(B_1 \mid Y)$ in a similar fashion:

$$P(B_1 \mid Y) = \frac{P(B_1 \cap Y)}{P(Y)} = \frac{P(Y \mid B_1)\,P(B_1)}{P(Y)} = \frac{5/9 * 0.5}{5/9 * 0.5 + 7/12 * 0.5} = 0.4878...$$