Conditional independence exists for two events $A$, $B$ given an event $C$ when: $P(A,B|C) = P(A|C)\cdot P(B|C)$

Bayes Theorem for two Events $E, F$: $P(E│F)= \frac{P(F|E) \cdot P(E)}{P(F)}$

I deduced that from a relationship between the events $X,Y,Z$ when $$P(X,Y,Z) = P(X)\cdot P(Y|Z)\cdot P(Z|X)$$ (a Bayesian network would look like $X \longrightarrow Z \longrightarrow Y$) there is a conditional independence for X and Y given Z : $$P(X,Y|Z)= \frac{P(X,Y,Z)}{P(Z)} = \frac{P(X)\cdot P(Z|X)}{P(Z)}\cdot P(Y|Z) = P(X|Z)\cdot P(Y|Z)$$

Meaning that if we can observe the event $Z$ and it holds, that both $X$ and $Y$ are only dependent on $Z$ and therefore conditionally independent from each other. But when I examined the case for when Z is not given($Z$ could occur $P(Z)$or not $P(\overline{Z})$), I don't know how to proceed to show dependency between X and Y:

$$P(X,Y) = P(X,Y,Z)+P(X,Y,\overline{Z}) = \\ P(X)\cdot P(Y|Z)\cdot P(Z|X) + P(X)\cdot P(Y|\overline{Z})\cdot P(\overline{Z}|X) = \space ?$$

After reading Did's comment, I made some changes to the Question

I changed the use of the notations from variables to events, since I seem to have abused them and created confusion. I believe it makes things simpler too. So, for an event $A$, $P(A)$ is the probability that $A$ occurs. $P(A, B) = P(A \cap B)$ is the prob. that A and B occur etc. .

I watched a video to explain different dependency relationships and it said for the structure shown above, from $P(X,Y) = P(X) \cdot P(Y|X)$ we can deduce that Y is dependent on X, if Z is not given (Video, in german), which is wrong.

My question in the first place was, how can I prove that Y and X are somehow dependent if we are not able to observe Z.

  • 1
    $\begingroup$ Are $X,Y,Z$ random variables? If so then what exactly is meant by expressions like $P(X,Y,Z)$, $P(X)$, $P(Y\mid Z)$ et cetera? If not then are $X,Y,Z$ values that can be taken by discrete random variables? $\endgroup$ – drhab Aug 10 '17 at 16:50
  • 1
    $\begingroup$ @Typhon Is one of the options that I gave correct? I am familiar with probability but not with the expressions. $\endgroup$ – drhab Aug 10 '17 at 17:05
  • 1
    $\begingroup$ @Typhon "...the probability that $X$ occurs".. Well, then we are not dealing with random variables $X,Y,Z$ but with events $X,Y,Z$ right? This $X,Y,Z$ almost always denote random variables. $\endgroup$ – drhab Aug 10 '17 at 17:12
  • 1
    $\begingroup$ @drhab If I may... Your questions are quite legitimate (and the reactions by user Typhon merely reveal a shaky knowledge of these matters). It is an unfortunate fact of life that similar abuses of notations are routinely used (for some of their devastating effects, see the many questions on the site by confused users). Here, $X$, $Y$, $Z$ are random variables, say discrete, and the assertion $$P(X,Y,Z)=P(X)\cdot P(Y|Z)\cdot P(Z|X)$$ actually means that, for every $(x,y,z)$, $$P(X=x,Y=y,Z=z)=P(X=x)\cdot P(Y=y|Z=z)\cdot P(Z=z|X=x)$$ Similarly for the other abuses of notation in the question. $\endgroup$ – Did Aug 11 '17 at 11:20
  • 1
    $\begingroup$ @Phise "The result should be $P(X)\cdot P(Y|X)$ to prove that Y is dependent on X when Z is not given" For your interest, the relation $$P(X,Y)=P(X)\cdot P(Y|X)$$ always hold hence it cannot help to prove any (in)dependence whatsoever. This is not the only mystery on this page, others being what you are really asking to show and how the accepted post is supposed to be an answer. Note that it may well happen that $X$ and $Y$ are independent, even with the dependence structure $$X\longrightarrow Z \longrightarrow Y$$ that you see to be interested in. $\endgroup$ – Did Aug 11 '17 at 11:27

Hint: $\Pr(Y|Z)=\Pr(Y|XZ)$. Consequently, $\Pr(Y|Z)\Pr(Z|X)=\Pr(Y|XZ)\Pr(Z|X)=\Pr(YZ|X)$

  • $\begingroup$ I can't quite follow. With that transformation I would get back to $$P(X)* \sum_Z P(Y|Z)*P(Z|X) = P(X)* \sum_Z P(X,Y,Z)* \frac{1}{P(X)} = \sum_Z P(X,Y,Z)$$ But I would like to move forward to $P(X)*P(Y|X)$ $\endgroup$ – Phise Aug 10 '17 at 17:18
  • $\begingroup$ $\sum_{Z} \Pr{(Y,Z|X)} = \Pr{(Y|X)}.$ $\endgroup$ – Math Lover Aug 10 '17 at 17:22
  • $\begingroup$ Ah. I am sorry, the last part where you wrote $ ... P(YZ|X) = \frac{P(X,Y,Z)}{P(X)}$ confused me. I am still wondering why $\sum_Z P(Y,Z|X) = P(Y|X)$, could you explain that part ? $\endgroup$ – Phise Aug 10 '17 at 17:40
  • $\begingroup$ That's why I wrote $\Pr(YZ|X)=\frac{\Pr(XYZ)}{\Pr(X)}$. Since $\sum_{B}\Pr(A,B) = \Pr(A)$, $\sum_{Z} \Pr{(XYZ)}=\Pr(XY)$. Consequently, $\sum_{Z} \Pr{(YZ|X)}=\frac{1}{\Pr{(X)}} \sum_{Z} \Pr{(XYZ)}=\frac{\Pr(XY)}{\Pr(X)}=\Pr(Y|X)$. $\endgroup$ – Math Lover Aug 10 '17 at 17:44
  • $\begingroup$ Got it, thanks for your patience and your great answer! (Don't have enough rep though to rate your answer) $\endgroup$ – Phise Aug 10 '17 at 17:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.