27
$\begingroup$

For a family of random variables, I was wondering if independence and conditional independence under any condition among them imply each other?

If not, can these two concepts imply one another under some special cases?

$\endgroup$
  • $\begingroup$ The question as you have stated it is vague. Is there some problem or result that is motivating this question? $\endgroup$ – svenkatr Feb 16 '11 at 22:31
  • $\begingroup$ @svenkatr: Is it correct that independent random variables may not be independent conditional on some conditions? Independence conditional on some conditions may not imply independence. So I was wondering: (1) if Independence conditional on any condition and independence may imply each other; (2) what are some useful cases where one can imply the other? $\endgroup$ – Tim Feb 16 '11 at 23:25
  • 1
    $\begingroup$ What is independence under any condition? If you mean independence under every possible conditioning, the condition is rarely fulfilled... For example, random variables $x$ and $y$ are not independent conditionally on $[x<y]$ if you exclude some very degenerate cases. $\endgroup$ – Did Feb 17 '11 at 11:34
27
$\begingroup$

Independence does not imply conditional independence: for instance, independent random variables are rarely independent conditionally on their sum or on their maximum.

Conditional independence does not imply independence: for instance, conditionally independent random variables uniform on $(0,u)$ where $u$ is uniform on $(0,1)$ are not independent.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ I did not understood the second example. I am unable to visualize random variables uniform on an interval in which one extreme of the interval is itself a random variable? $\endgroup$ – raffaem Apr 19 '19 at 16:58
59
$\begingroup$

I like to interpret these two concepts as follows:

  • Events $A,B$ are independent if knowing that $A$ happened would not tell you anything about whether $B$ happened (or vice versa). For instance, suppose you were considering betting some money on event $B$. Some insider comes along and offers to pass you information (for a fee) about whether or not $A$ happened. Saying $A,B$ are independent is to say that this inside information would be utterly irrelevant, and you wouldn't pay any amount of money for it.

  • Events $A,B$ are conditionally independent given a third event $C$ means the following: Suppose you already know that $C$ has happened. Then knowing whether $A$ happened would not convey any further information about whether $B$ happened - any relevant information that might be conveyed by $A$ is already known to you, because you know that $C$ happened.

To see independence does not imply conditional independence, one of my favorite simple counterexamples works. Flip two fair coins. Let $A$ be the event that the first coin is heads, $B$ the event that the second coin is heads, $C$ the event that the two coins are the same (both heads or both tails). Clearly $A$ and $B$ are independent, but they are not conditionally independent given $C$ - if you know that $C$ has happened, then knowing $A$ tells you a lot about $B$ (indeed, it would tell you that $B$ is guaranteed). If you want an example with random variables, consider the indicators $1_A, 1_B, 1_C$.

Of interest here is that $A,B,C$ are pairwise independent but not mutually independent (since any two determine the third).

A nice counterexample in the other direction is the following. We have a bag containing two identical-looking coins. One of them (coin #1) is biased so that it comes up heads 99% of the time, and coin #2 comes up tails 99% of the time. We will draw a coin from the bag at random, and then flip it twice. Let $A$ be the event that the first flip is heads, $B$ the event that the second flip is heads. These are clearly not independent: you can do a calculation if you like, but the idea is that if the first flip was heads, it is strong evidence that you drew coin #1, and therefore the second flip is far more likely to be heads.

But let $C$ be the event that coin #1 was drawn (where $P(C)=1/2$). Now $A$ and $B$ are conditionally independent given $C$: if you know that $C$ happened, then you are just doing an experiment where you take a 99%-heads coin and flip it twice. Whether the first flip is heads or tails, the coin has no memory so the probability of the second flip being heads is still 99%. So if you already know which coin you have, then knowing how the first flip came out is of no further help in predicting the second flip.

| cite | improve this answer | |
$\endgroup$
  • 5
    $\begingroup$ This is a great answer. Should be the accepted one in my opinion. It would be great if there's an intuitive real world example for the inverse as well, where a conditional independence does not imply independence? $\endgroup$ – dev_nut Dec 16 '14 at 19:21
  • 7
    $\begingroup$ @dev_nut: Trivial example: $A$ and $A$ are conditionally independent given $A$, but not independent. $\endgroup$ – Nate Eldredge Dec 16 '14 at 19:33
  • 1
    $\begingroup$ Thanks. It's trivial, but real world? :). Not as intuitive as your answer. $\endgroup$ – dev_nut Dec 16 '14 at 20:32
  • $\begingroup$ @NateEldredge any more obvious example? This trivial one is kind of hard to fathom using intuition ;-) $\endgroup$ – Sнаđошƒаӽ Oct 27 '15 at 16:53
  • 1
    $\begingroup$ I've added another example. $\endgroup$ – Nate Eldredge Mar 13 '18 at 21:19
1
$\begingroup$

If $(X,Y,Z)$ are (mutually) independent then $X$ is independent of $Y$ conditional on $Z$. Proof:

$$ f_{x,y|z}=f_{x,y,z}/f_z=f_{x}f_{y}f_z/f_z=f_xf_y=f_{x|z}f_{y|z}. $$

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.