5
$\begingroup$

I am trying to understand (in)dependence and conditional (in)dependence. As far as I know conditional independence does not imply unconditional independence and vice versa. Assuming that I have three random binary variables A, B and C, I can easily find such a case when A and B are independent, but conditionally dependent given C. However I am struggling to find an example of joint distribution of this three variables A,B and C such that A and B are dependent, but given C are independent. Can someone guide me preferably by giving such an example?

$\endgroup$
4
$\begingroup$

Consider two internet subscribers, that are serviced by the same provider and from the same local network node. For a specific time period, define the Bernoulli random variables $A=$"Subscriber A calls the support center =$1$, doesn't call =$0$", and $B=$"Subscriber B calls the support center =$1$, doesn't call =$0$". These random variables are not independent. There are many events that may make subscriber A call the call center, totally independent of B (like problems with A's home router), and likewise for subscriber B. But there is also a common possible cause: the local network node that services both is down. So $P(A\mid B) \neq P(A)$ , $P(B\mid A) \neq P(B)$, and $$P(A,B) \neq P(A)P(B) \qquad [1]$$

Note that the fact that the node may be down does not make certain that the subscriber will call - they may be out of their homes, or whatever. But it certainly affects the probabilities of calling or not.

Define now a third random variable, $C=$"Local network node is down =$1$, is up=$0$".

Given $C=0$ then $A$ and $B$ become independent: you know that the possible common source of affecting the probabilities of them calling in is absent, so if they call, they will do it due to independent events. So

$$P(A,B \mid C=0) = P(A \mid C=0)P(B \mid C=0)\qquad [2]$$

Assume now that $C=1$, i.e. the network node is down. Knowledge of this event too makes $A$ and $B$ independent:

Consider $P(B \mid \{A=0, C=1\})$ "probability of B calling given that A didn't call and that the node is down". Does in this case matter for what B will probabilistically do, whether A didn't call? No: what A does affects the probabilities of what B will do because it conveys some possible information on whether the node is down or not. If we already know that the node is down, then the behavior of A does not "tell us" anything additional related to B's behavior, and so does not affect the probabilities related to B, again, given that the node is down. So $P(B \mid \{A=0, C=1\}) = P(B \mid C=1)$.
Likewise for the case $P(B \mid \{A=1, C=1\}) = P(B \mid C=1)$. So we conclude that $P(B \mid \{A, C=1\}) = P(B \mid C=1\})$.
But then since by definition $P( A, B \mid C=1) = P (B \mid \{A, C=1\}) P(A \mid C=1)$ we obtain $$P( A, B \mid C=1) = P (B \mid C=1) P(A \mid C=1)\qquad [3]$$

Combining eq $[2]$ and $[3]$ we arrive at

$$P( A, B \mid C) = P (B \mid C) P(A \mid C)\qquad [4]$$

The random variables $A$ and $B$ are unconditionally dependent, but become independent conditional on the random variable $C$.
Intuitively and informally, conditional independence arises when the "reason for the dependence" is the conditioning factor, because then the uncertainty with respect to this "reason" is "taken out" of the distributions of the variables, leaving only the "independent parts".

$\endgroup$
  • $\begingroup$ I love the example, I often need them to understand 'simple' concepts and this certainly helped! $\endgroup$ – Tommaso Guerrini Jan 19 '17 at 9:53
  • 1
    $\begingroup$ @TommasoGuerrini Glad I could provide some intuition to you. To be honest this is one of my favorite self-made examples too. $\endgroup$ – Alecos Papadopoulos Jan 19 '17 at 13:16
3
$\begingroup$

Here is a boring example. Toss a fair coin. Let $A$ be the event we got a head, and $B$ the event we got a tail. Then $A$ and $B$ are not independent. Let $C$ be the event we got a head.

Then $\Pr((A\cap B)|C)=0=\Pr(A|C)\Pr(B|C)$, since $\Pr(B|C)=0$.

$\endgroup$
1
$\begingroup$

Another example is a Markov process, in which $C$ is the "middle" variable. Suppose that $A \to C \to B$ (for example, given three iid $X,Y,Z$, we define $A=X$, $C=A+Y$, $B = C + Z$).

In this case, we have $P(B | A C) = P(B|C)$ (definition of markov property), so $P( A B | C) = P (B | A C) P(A |C) = P (B | C) P(A |C) $

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.