1
$\begingroup$

I'm in a pre rigorous phase but as a programmer I need to know certain things on a case by case basis to do certain very particular things.

Therefore, I was hoping for validation of why the following, observed in a book about bayesian networks, is true. That being:

Let $\mathcal{G}$ be a bayesian network over $X_1, ..., X_n$. We say that a distribution $P_\beta$ over the same space factorizes according to $\mathcal{G}$ if $P_\beta$ can be expressed a product:

$$\mathbb{P}_\beta (X_1, ..., X_n) = \prod\limits_{i=1}^n\mathbb{P}(X_i \mid \mathbf{Pa}_{X_i})$$

where $\mathbf{Pa}_{X_i}$ is a vector of the observations of the parent nodes of $X_i$

I was wondering where we get the product formula for $\mathbb{P}_\beta$ from. Does it follow directly from the chain rule for probability? If so, can so please do a derivation.

$\endgroup$
2
$\begingroup$

The decomposition into the product expresses that each $X_i$ is conditionally independent from all other $X_j$ for all $j \neq i$ given the parents of $X_i$ in the Bayesian network.

As a basic example, consider the Bayesian network consisting of four nodes: $X_1 \rightarrow X_2 \rightarrow X_3 \rightarrow X_4.$

Then, $$P(X_1, X_2, X_3, X_4) = P(X_1)P(X_2 | X_1)P(X_3|X_1,X_2)P(X_4|X_1,X_2,X_3)$$ $$= P(X_1)P(X_2 | X_1)P(X_3|X_2)P(X_4|X_3).$$

Why is each node conditionally independent of all other nodes given its parents? This is from the definition of Bayesian networks.

$\endgroup$
0
$\begingroup$

Pearl is saying that that if the distribution factorizes in this way, then $\mathcal G$ is a possible structure.

$\endgroup$
  • $\begingroup$ Thanks for your answer. Not quite sure I get what you are saying though $\endgroup$ – theideasmith Apr 4 '17 at 22:09
  • $\begingroup$ @theideasmith Your question is about a definition. There is no proof for it. It's a starting point — an assumption — from which to build other things. $\endgroup$ – Neil G Apr 4 '17 at 22:11
  • $\begingroup$ That's what I thought. But I mean in general if $X_1, X_2, X_3..$ are independent then $P(\mathbf{X}) = \prod X_i$, right? So I was wondering if this definition follows from that fact? $\endgroup$ – theideasmith Apr 4 '17 at 22:23
  • $\begingroup$ @theideasmith Yes, your first statement is correct. If the $X_i$ are independent, then they can be factored according to a totally disconnected graph. In your second statement, you have it backwards. The point is to study structures in which there are some dependencies, but also many conditional independencies. $\endgroup$ – Neil G Apr 4 '17 at 22:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.