Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

My understanding right now is that an example of conditional independence would be:

If two people live in the same city, the probability that person A gets home in time for dinner, and the probability that person B gets home in time for dinner are independent; that is, we wouldn't expect one to have an affect on the other. But if a snow storm hits the city and introduces a probability C that traffic will be at a stand still, you would expect that the probability of both A getting home in time for dinner and B getting home in time for dinner, would change.

If this is a correct understanding, I guess I still don't understand what exactly conditional independence is, or what it does for us (why does it have a separate name, as opposed to just compounded probabilities), and if this isn't a correct understanding, could someone please provide an example with an explanation?

share|cite|improve this question
up vote 33 down vote accepted

The scenario you describe provides a good example for conditional independence, though you haven't quite described it as such. As the Wikipedia article puts it, "$R$ and $B$ are conditionally independent [given $Y$] if and only if, given knowledge of whether $Y$ occurs, knowledge of whether $R$ occurs provides no information on the likelihood of $B$ occurring, and knowledge of whether $B$ occurs provides no information on the likehood of $R$ occurring". In this case, $R$ and $B$ are the events of persons A and B getting home in time for dinner, and $Y$ is the event of a snow storm hitting the city. Certainly the probabilities of $R$ and $B$ will depend on whether $Y$ occurs. However, just as it's plausible to assume that if these two people have nothing to do with each other their probabilities of getting home in time are independent, it's also plausible to assume that, while they will both have a lower probability of getting home in time if a snow storm hits, these lower probabilities will nevertheless still be independent of each other. That is, if you already know that a snow storm is raging and I tell you that person A is getting home late, that gives you no new information about whether person B is getting home late. You're getting information on that from the fact that there's a snow storm, but given that fact, the fact that A is getting home late doesn't make it more or less likely that B is getting home late, too. So conditional independence is the same as normal independence, but restricted to the case where you know that a certain condition is or isn't fulfilled. Not only can you not find out about A by finding out about B in general (normal independence), but you also can't do so under the condition that there's a snow storm (conditional independence).

An example of events that are independent but not conditionally dependent would be: You randomly sample two people A and B from a large population and consider the probabilities that they will get home in time. Without any further knowledge, you might plausibly assume that these probabilities are independent. Now you introduce event $Y$, which occurs if the two people live in the same neighbourhood (however that might be defined). If you know that $Y$ occurred and I tell you that A is getting home late, then that would tend to increase the probability that B is also getting home late, since they live in the same neighbourhood and any traffic-related causes of A getting home late might also delay B. So in this case the probabilities of A and B getting home in time are not conditionally independent given $Y$, since once you know that $Y$ occurred, you are able to gain information about the probability of B getting home in time by finding out whether A is getting home in time.

Strictly speaking, this scenario only works if there's always the same amount of traffic delay in the city overall and it just moves to different neighbourhoods. If that's not the case, then it wouldn't be correct to assume independence between the two probabilities, since the fact that one of the two is getting home late would already make it somewhat likelier that there's heavy traffic in the city in general, even without knowing that they live in the same neighbourhood.

To give a precise example: Say you roll a blue die and a red die. The two results are independent of each other. Now you tell me that the blue result isn't a $6$ and the red result isn't a $1$. You've given me new information, but that hasn't affected the independence of the results. By taking a look at the blue die, I can't gain any knowledge about the red die; after I look at the blue die I will still have a probability of $1/5$ for each number on the red die except $1$. So the probabilities for the results are conditionally independent given the information you've given me. But if instead you tell me that the sum of the two results is even, this allows me to learn a lot about the red die by looking at the blue die. For instance, if I see a $3$ on the blue die, the red die can only be $1$, $3$ or $5$. So in this case the probabilities for the results are not conditionally independent given this other information that you've given me. This also underscores that conditional independence is always relative to the given condition -- in this case, the results of the dice rolls are conditionally independent with respect to the event "the blue result is not $6$ and the red result is not $1$", but they're not conditionally independent with respect to the event "the sum of the results is even".

share|cite|improve this answer
11  
This is an excellent answer and very helpful! Thank you! – Ryan Feb 21 '11 at 19:23
    
I don't understand how the first two examples are qualitatively different, or to be more precise, how A is conditionally independent of B given Y in the snowstorm case. If I know that a snow storm has occured, and then I know that A did not make it home on time, am I not likely to increase the probability that B does not make it home on time, since clearly the storm is having an effect on everyone in the city. And vice-versa if A did make it home on time, does not the probability that B make it home on time increase? – zenna Oct 4 '12 at 5:45
    
@zenna What if B works and lives in the same building? – Mike Wierzbicki Nov 12 '13 at 13:48

I will compass here to exemplify a case not yet discussed: an example of dependence but conditional INdependence. Please let me know of improvements or errors.

In keeping with your OP, where you defined $A$ and $B$ to live in the same city, let
$Pr(A)$ = probability that person A gets home in time for dinner and
$Pr(B)$ = the probability that person B gets home in time for dinner.
Here, I assume that $A$ & $B$ live in the same neighbourhood.

Let $E$ be the event of a mandatory city-wide evacuation due to the menace of a falling satellite.

In line with user joriki's 2nd paragraph, $Pr(A)$ and $Pr(B)$ are conceivably dependent.

However, given/on the condition of $E$, both persons $A$ and $B$ probably won't be coming home tonight. If the satellite crashes before dinnertime, then mayhap. If it still hasn't crashed, then probably not. In both cases, any information on person A's time of return reveals nothing about person B's. In other words, $Pr(A|E)$ and $Pr(B|E)$ will be conditionally independent.

share|cite|improve this answer
1  
The assertions that "Pr(A) and Pr(B) are conceivably dependent" and that "Pr(A|E) and Pr(B|E) will be conditionally independent" have no meaning I would be aware of. For example, Pr(A) and Pr(B) are numbers, as such they are neither dependent nor independent. – Did Nov 12 '13 at 13:55

The events would be conditionally independent, for example, if A always comes home by car and B commutes by subway/metro train. Assuming subway is not affected by a snow storm but road transport is, it can be said that they are conditionally independent.

share|cite|improve this answer

I actually find the above answers somewhat confusing. By definition, events A and B are independent if

$p(A|B)=p(A)$.

If this is truly the case (as stated in the question), there is no need for conditional independence. These events are independent given any condition.

However if A and B are not independent, they might still be conditionally independent given Z. This is the case if

$p(A|B,Z) = p(A|Z)$.

A nice example paraphrased from Norman Fenton's website: if Alice (A) and Bob (B) both flip the same coin, but that coin might be biased, we cannot say

$p(A=H|B=H) = p(A=H)$

(i.e. that they are independent) because if we see Bob flips heads, it is more likely to be biased towards heads, and hence the left probability should be higher. However if we denote Z as the event "the coin is biased towards heads", then

$p(A=H|B=H,Z)=p(A=H|Z)$

we can remove Bob from the equation because we know the coin is biased. Given the fact that the coin is biased, the two flips are conditionally independent.

Returning to the snowstorm affecting the travel times of two people, this is an example that shows you might think A and B are independent (they take totally different routes home), but actually they are only conditionally independent given the snowstorm, not truly independent (or "statistically independent").

This notion of not statistically independent but conditionally independent is closely related to "correlation does not imply causation" (specifically the third cause fallacy). It wasn't the fact that Bob flipped heads that caused Alice to flip heads, it was the fact that the coin was biased. Similarly, one person getting home late doesn't cause another person to get home late, but those times might be correlated in the case of a snowstorm.

share|cite|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.