23
$\begingroup$

What is some intuitive insight regarding the conditional probability definition: $P(A\mid B) = \large \frac{P(A \cap B)}{P(B)}$ ? I am looking for an intuitive motivation. My textbook merely gives a definition, but no true development of that definition. Hopefully that's not too much to ask.

$\endgroup$
  • $\begingroup$ Informally, the chances A occurs given that B has already occurred is taken by finding the probability that both A and B occur, and divide by the chance B has already occurred. $\endgroup$ – muzzlator Feb 4 '13 at 15:34
  • 2
    $\begingroup$ Drawing a Venn diagram for $A$ and $B$ and interpreting probabilities as areas may help. If you're given that $B$ has occurred, then you may restrict your attention to the new sample space $B$. The probability that $A$ occurs given $B$ is the ratio of the area of $A\cap B$ to the area of $B$. $\endgroup$ – David Mitra Feb 4 '13 at 15:35
33
$\begingroup$

Consider probabilities as proportions. To say that something has probability one-sixth is to say it occurs one-sixth of the time (this is only one interpretation: it suits our purposes and our intuition, so let's not worry too much about what it means philosophically). Often we calculate probabilities simply by dividing the number of possibilities in which our event of interest occurs, by the number of possibilities total – e.g. to calculate the odds of throwing an even number on a six-sided dice, we calculate $3/6$. (This works because each of the possibilities we are counting is equally likely, by assumption).

Now let's say we want to work out how often $A$ occurs, given that we know $B$ has occurred. Well, we need to find the occurrences of $A$ in this scenario, and divide by the total number of possibilities. When we know $B$ occurred, the occurrences of $A$ are all and exactly those situations in which both $A$ and $B$ occur, and since we're assuming $B$ occurred, the total number of possibilities are reduced to only those where that happened.

Hence \[\mathbb P(A\mid B) = \frac{\text{# occurrences of A and B}}{\text{# occurrences of B}} = \frac{\mathbb P(A \cap B)}{\mathbb P(B)}\]

because the "total number of possibilities" in the expressions for $\mathbb P(B)$ and $\mathbb P(A \cap B)$ cancel.

Essentially, what we are doing is focussing on a particular subsection of the potential events, and considering what proportion of that subsection satisfies whatever property you're interested in (think of Venn diagrams). So, for example, given that your roll result was even, on a six-sided die, it is less likely to be less than $4$, because half the numbers $\{1,2,3,4,5,6\}$ are less than $4$ but only a third of the numbers in our subsection $\{2,4,6\}$ are.

$\endgroup$
  • 1
    $\begingroup$ "this is only one interpretation: it suits our purposes and our intuition, so let's not worry too much about what it means philosophically" --- What does it mean philosophically? $\endgroup$ – WorldGov Jul 21 '18 at 8:49
  • $\begingroup$ Well as I understand it the frequentist interpretation is to embed each event in some kind of series of many experiments and look at the long-run average occurrences of the outcomes. This is easy to do with coin flips or dice rolls but it's less clear what the appropriate series of experiments would be if you were doing something like asking what the probability was that the Cuban Missile Crisis would lead to nuclear war (say). $\endgroup$ – Ben Millwood Jul 22 '18 at 10:19
  • $\begingroup$ The Bayesian interpretation sidesteps that problem by saying that probability is just a measure of your lack of information about what the outcome will be (and is therefore subject-dependent), and to say that something occurs with probability $p$ is just to say that you'd be willing (in some abstract idealized sense) to pay £$p$ (and no more) for the opportunity to win £1 if the thing happens. $\endgroup$ – Ben Millwood Jul 22 '18 at 10:24
8
$\begingroup$

Think about this: if $B$ is very unlikely but when it happens $A$ becomes likely then $P(A \text{ and } B)$ is small while $P(A|B)$ is large.

I am extremely unlikely to win the lottery jackpot this weekend ($B$) but if I do I am likely to become a millionaire ($A$), so the probability I win the lottery jackpot and then become a millionaire $P(A \text{ and } B)$ is small, but the probability I become a millionaire if I win the lottery jackpot $P(A|B)$ is high.

$\endgroup$
5
$\begingroup$

Let $\Omega$ be your space of possibilities. Then $B$ is a subset of $\Omega$. The probability $\mathbb P$ induces a probability for events lying in $B$. Namely $\mathbb P_B = \frac{\mathbb P}{\mathbb P(B)}$. You can check the important fact that $\mathbb P_B(B) = 1$.

Then the event $A \cap B$ seen as an event in $B$ has probability $\mathbb P_B(A \cap B)$.

So I would say that this is just a base change from $\Omega$ to $B$.

$\endgroup$
4
$\begingroup$

You might want to multiply through by $P(B)$ and try to understand

$$ P(A, B) = P(A|B) P(B). $$

This says that the probability that $A$ and $B$ both happen can be computed by first taking the probability that $B$ happens, then multiply by the probability that $A$ will happen given that $B$ happens.

$\endgroup$
4
$\begingroup$

Suppose that I reach into my pocket, pick a coin at random, and toss it. $A$ is the event the coin comes up heads, and $B$ is the event the coin has heads on both sides. Clearly $P(A\mid B)=1$.

As it happens, there are $10$ coins in my pocket, and only one of them has heads on both sides. When I reach blindly into my pocket and pull out a coin, there’s only one chance in $10$ that I get the two-headed coin, so $P(A~\mathbf{and}~B)$ can be at most $\frac1{10}$, the probability of $B$ alone. In this example $P(A\mid B)$ clearly cannot be equal to $P(A~\mathbf{and}~B)$, and it’s easy to come up with many similar examples. Such examples don’t show what the right calculation is, but they do show clearly that in general $P(A\mid B)\ne P(A~\mathbf{and}~B)$; other answers have explained the correct formula.

$\endgroup$
  • $\begingroup$ nice example for understanding of concept!!! $\endgroup$ – Fennekin May 14 '17 at 7:01
0
$\begingroup$

It follows from the following insight:

The probability that A and B happen is the probability that B happens, multiplied by the probability that A happens given that B has already happened.

or in symbols

$$P(A, B) = P(A|B)P(B)$$

This rearranges into

$$P(A|B) = \frac{P(A, B)}{P(B)}$$

$\endgroup$
0
$\begingroup$

Think of it this way. What is the probability that event a and event b happens?

To get this you have to understand that in independent joint distributions, p(a,b) = p(a AND b) = p(a)*p(b). e.g. the probability you get a heads then a tail on two coin flips is the probability of getting a heads on first flip multiplied by the probability of getting a tails on the second flip. 1/2 * 1/2 = 1/4.

So going back to your example. Since a is dependent on b, then the probability of a and b, p(a AND b), is equal to probability that b happens, which is independent, and the probability that a happens, which is dependent on b, p(a|b).

Therefore if you plug it into the independent joint distributions equation I noted above, you get

p(a AND b) = p(b) * p(a|b).

So if you move things around you get,

p(a|b) = p(a AND B)/p(b)

Or if capitalize it as you did:

P(A|B) = P(A AND B) / P(B)

$\endgroup$
0
$\begingroup$

In summary, the essence of conditional probability is in every probability calculation, find the Whole (i.e. universal set), find the Part (intersection between the event you are trying to calculate the probability and the universal set) and divide the Part by the Whole to find the probability of given event (i.e the comparative relation between Part and Whole).

Let A be any given event and U be the set of all possible events or outcomes:

$ P(A)=\frac{P(A)}{P(U)} = \frac{P(A \cap U)}{P(U)} = P(A|U) $

If you are struggling to see the full equivalence above, remember that in set theory $ A \cap B = A $ when A is contained in B, which is always the case for any set when are trying to find the intersection with the Universal set (U).

Why do we need to divide?

Proportions: Finding the likelihood or probability of some event can be interpreted as a proportion problem, which implies the existence of a Part and a Whole. The way that you find how much a Part represent from the Whole is by dividing the Part by the Whole: $ \frac{Part}{Whole} $ (if you are still uncomfortable with the concept and for some reason can't visualize why division would provide you with such information, just take it as a rule for now, at some point the concept will click). So whenever you are thinking about the probability or likelihood of an event always think about the Part and the Whole, which leads to the next question: How do we find the Part and the Whole? We will need to understand how we represent events to answer that question.

Sets, Subsets and Cardinalities: An event is a set of outcomes, the cardinality of a set is the measure of the size of that same set. Now, for the sake of our probabilistic theory discussion, you will see that a Part is always a subset of the Whole (you may be thinking of some edge cases that do not fit this rule, keep reading and you will see that this idea indeed holds true for all cases).

The first step to find a probability of any event is to find the set that represents the Whole (i.e. the scope of all the possible outcomes, when you roll a die of 6 sides the Whole set can be represented as the numbers from 1 through 6, since they form all the possible outcomes), the second step is to find the Part (i.e. the set containing the subset of outcomes that you are trying to find the likelihood of occurrence).

Every probability calculation can be seen with the lens of a conditional probability (What!? Keep reading :) )

Let A be any given event and U be the set of all possible events or outcomes:

$ P(A)=\frac{P(A)}{P(U)} = \frac{P(A \cap U)}{P(U)} = P(A|U) $

If you are struggling to see the full equivalence above, remember that in set theory A and B = A when A is contained in B, which is always the case for any set when are trying to find the intersection with the Universal set.

As an example, say we are interested in knowing the probability of getting an even number when rolling a die of 6 sides, you can always formulate the problem thinking about conditional probability, when we defined that we are going to be rolling a die of six sides we just implicitly defined the Whole set (i.e the possible outcomes are numbers from 1 to 6), the second piece of information is the event itself that we would like to obtain the probability (even number), that is the Part. An important subtlety here, the Part you are interested here in the first place, is not ALL the even numbers but the ones that make part of your Whole set (i.e. subset of the Whole). How do you find that subset? The intersection of the two sets? precisely.

So the intersection gives you the Part that you are interested in, and once you have the Whole and Part you divide the two and find the probability (frequency of occurrence, or the comparative relation between the Part and the Whole).

If the outcomes or event that you are trying to find the likelihood does not make any part of the Whole, in other words, if the intersection of both sets is empty then we know that the probability of such event is zero, the event is simply impossible to occur, there is no way you will get a 7 if you roll a die of 6 sides.

Always remember when you see P(A|B) we are already explicitly saying that B is the Whole or universal set, all you have to do now is to find the comparative relation between the Part and Whole..

To end, let me clarify a few common misconceptions:

  1. On the context of conditional probability, aside from the relationship between the Whole and Part, the probability of what you are judging to be the Whole set does not matter by itself! As an example, the probability of someone having a tumor is irrelevant, if what we are trying to find is the probability of someone having a benign tumor given the person is already diagnosed with a tumor.

  2. P(A|B) is not the same as P(B|A): The probability of someone having a benign tumor given the person is already diagnosed with a tumor, is not the same as the probability of someone being diagnosed with a tumor given the fact that the person has a benign tumor.

  3. Shouldn't P(A|B) be equal to P(A and B)? This is a common misinterpretation of what conditional probability means, we are not interested in the probability of A and B happening (i.e. $ \frac{P(A \cap B)}{P(U)} $), that's called joint probability and is a separate (and related) subject. Conditional probability means, now that B ALREADY HAPPENED, what is the probability of A also happening?

$\endgroup$
-1
$\begingroup$

As we are given that $B$ occurred, we reduce the sample space to $B$. Now that, we have reduced the sample space to $B$ we look at what is the probability $A$ has also occurred which leads to the intersection of both i.e $A \cap B$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.