6
$\begingroup$

I known that for give $x,y$, if we have
$$p(x,y)=p(x)p(y)$$
Then we call $x,y$ are independent.
For marginal independence, I found the definition here.

Random variable $x$ is marginal independent of random variable $y$ if: $$p(x|y)=p(x)$$

However, I cannot see the difference between them.

The question come to my mind when I want to figure out a question about Bayesian network. In 3-way Bayesian network, there are three nodes A,B,C in a common parent struct.

   A  
  | |  
 B   C

A is the parent of B and C. It says B and C are conditional independent given A, and I can see it because:
$$P(B|A,C)=\frac{P(A,B,C)}{P(A,C)}\\=\frac{P(A)P(B|A)P(C|A)}{P(A,C)}\\=P(B|A)$$
But, when the value of A is unknown, why $B,C$ are not independent?

$\endgroup$
  • $\begingroup$ Marginal independent is the same as independent. Conditionally independent is the same but every works after you condition on some certain event (here A). $\endgroup$ – Jimmy R. Oct 20 '17 at 7:08
  • $\begingroup$ One can only wonder why the author sees fit to rename "marginal independence" the independence property. This can only confuse things, especially in a teaching context. Really, I find this pedagogical choice rather detrimental. $\endgroup$ – Did Dec 19 '18 at 19:37
7
$\begingroup$

As Jimmy R points out, marginal independence is the same as traditional independence, and conditional independence is just the independence of two random variables given the value of another random variable.

To give one example for why $B$ and $C$ are not independent when $A$ is unknown, suppose that $A$ is a random variable detailing whether a coin is biased. If $A=Y$, the coin is biased and will always give heads (magically), and if $A=N$ then the coin is fair.

Now suppose $B$ and $C$ are random variables on the outcome of flipping the (possibly-biased) coin. If we know whether the coin is biased, then $B$ and $C$ are independent (as they're different coin flips) and we know the probability distributions for both (which are the same).

However, suppose we don't know the outcome of $A$, the coin may or may not be biased. Now suppose that the outcome of $B$ comes up tails. Then we know that the outcome of $C$ must be fair, as the coin must be fair, i.e. $A=N$ (else $B$ couldn't give us tails). Hence, without prior knowledge of $A$, we have that $B$ and $C$ are dependent on eachother.

We therefore say that $B$ and $C$ are conditionally independent on/given $A$ but are (marginally) dependent.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ B depends on A so how is A and B independent regardless of if the coin flip is bias? Also, how is A and B a different coin flip? Also You switched from using Y/N to T/F for A being a fair coin? $\endgroup$ – Michael Fulton Jun 1 '19 at 19:13
  • 1
    $\begingroup$ @MichaelFulton I see my notation got a little confused and I didn't notice. I'll amend it now, and if you still have a question after I've corrected it do let me know. $\endgroup$ – Alex W Jun 2 '19 at 13:47
1
$\begingroup$

But, when the value of A is unknown, why B,C are not independent?

The formulation of your problem points me toward a solution. Let's say they are independent(proof by contradiction).

If B and C are independent then $P(B, C) = P(B) P(C)$, and $P(A, B, C) = P(A|B, C) P(B,C)=P(A|B, C)P(B)P(C)=P(A)P(B)P(C)$. And the graph(two children share a commen parent) in your question doesn't work anymore.

I'd like to plug an intuitive instance. Let's consider that A is the age of a certain person and B is his/her reading ability and C stands for his/her hight. Knowing A decouples B and C, as obviously as anyone can see. But if A is not given, we can infer B by C or C according to B. If he/she is tall we can guess he/she is educated enough to read fluently and if someone's reading ability is high we can also conjecture that he/she most probably would not be as short as a child in kindergarten.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.