1
$\begingroup$

To ask my question I will present an example. Suppose we have $2$ boxes with yellow and blue balls. Box 1 has $5$ yellow and $4$ blue balls and Box 2 has $7$ yellow and $5$ blue balls. We draw a ball at random from one of the boxes ($50-50$ chance) and the ball is yellow. What is the probability of the ball originating from Box 1?

To solve this, I used conditional probability which I understand is wrong and the correct answer would be to use Bayes theorem, which I didn't even know about. However because the chance is $50-50$ for either box the results were very similar.

My question is, what is the difference between the two formulas?

$\endgroup$
1

1 Answer 1

4
$\begingroup$

First I explain why there is no real difference between both, then I tackle your example.

There is no fundamental difference between conditional probability and Bayes' theorem. Actually, Bayes' theorem is a simple rewrite of the definition of conditional probability.

Let $A$ and $B$ be two events.

The conditional probability of event $A$ knowing $B$ occurred is:

$$P(A \mid B) = \frac{P(A \cap B)}{P(B)}$$

Basically this formula restricts the sample space to what happens within event B, and normalize the new probability measure by dividing by $P(B)$ to ensure the sum is $1$ as it should be.

If you manipulate this expression, you find:

$$P(A \cap B) = P(A\mid B)\,P(B)$$

Of course, by symmetry, replace $A$ by $B$ and you get:

$$P(B \cap A) = P(B\mid A)\,P(A)$$

Now, Bayes' theorem simply merges these two expressions in a different form. In a way, it's not really a "theorem" because its proof is trivial. But its implication for statistics are so wide that it's called "theorem"

Bayes theorem:

$$P(A \mid B) = \frac{P(B \cap A)}{P(B)} = \frac{P(B \mid A)\,P(A)}{P(B)}$$

Asking the difference between Bayes' theorem and conditional probability is like asking the difference between these two equations: $$ x = \frac{a}{b} \quad \text{ and } \quad b\times x = a $$

Hope this helps


Edit:

to tackle your example:

Let $B_1$ the event "the ball is drawn from box 1" and $B_2$ the event "the ball is drawn from box 2". We know that $P(B_1) = P(B_2) = 0.5$.

In box 1, there are 5 yellow and 4 blue balls so if $Y$ is the event "we draw a yellow ball", then $P(Y \mid B_1) = 5 / (4+5) = 5/9$.

Likewise for box 2, $P(Y \mid B_2) = 7 / (7 + 5) = 7 / 12$

With this setup, we have:

$$P(Y) = P(Y \mid B_1)\,P(B_1) + P(Y \mid B_2)\,P(B_2)$$ $$P(Y) = 5/9 * 0.5 + 7/12 * 0.5 \approx 0.56944444$$

This formula I used is called "law of total probability". It can be used when two (or more) events partition the sample space $\Omega$. That is: $B_1 \cap B_2 = \emptyset$ and $B_1 \cup B_2 = \Omega$.

Now, you can compute $P(B_1 \mid Y)$ in a similar fashion:

$$P(B_1 \mid Y) = \frac{P(B_1 \cap Y)}{P(Y)} = \frac{P(Y \mid B_1)\,P(B_1)}{P(Y)} = \frac{5/9 * 0.5}{5/9 * 0.5 + 7/12 * 0.5} = 0.4878...$$

$\endgroup$
5
  • $\begingroup$ +1, if only for the recapitulation at the end. $\endgroup$
    – Did
    Apr 27, 2018 at 10:07
  • 1
    $\begingroup$ @Did You wrote "+1", but didn't upvote. What does your comment mean then? $\endgroup$
    – Git Gud
    Apr 27, 2018 at 10:12
  • $\begingroup$ Thank you for this answer. In the case of my example, the solutions are not exactly the same when using conditional probability or bayes theorem. For C.P. I get 0.486 but for Bayes I get 0.49 $\endgroup$
    – System
    Apr 27, 2018 at 10:18
  • $\begingroup$ Done now. $ $ $ $ $\endgroup$
    – Did
    Apr 27, 2018 at 10:49
  • $\begingroup$ @System, the answer should be the same whatever means you use if your calculation is right. I find 0.4878... as shown in my edit. Maybe you did a small mistake or your calculator is too approximate? In particular, you should use the exact value of $P(Y)$ to compute the last fraction, not the approximate value. Otherwise, you accumulate approximations. In any case, remember to mark this answer as accepted if it answers your question. Best $\endgroup$
    – Julien__
    Apr 27, 2018 at 11:58

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .