1
$\begingroup$

I am currently reading about Independence Maps in Bayesian Reasoning and Machine Learning by David Barber and I'm trying to understand example 4.4, which is:

Example 4.4 (page 77) Consider the distribution (class) defined on variables $t_1,t_2,y_1,y_2$: $$p(t_1,t_2,y_1,y_2)=\\ p(t_1)p(t_2)\sum_h p(y_1|t_1,h)p(y_2|t_2,h)p(h)$$ In this case the list of all independence statements (for all distribution instances consistent with $p$) is $$\mathcal{L}_P=\{y_1 \perp\kern-5pt\perp t_2 | t_1, y_2 \perp\kern-5pt\perp t_1 | t_2, t_2 \perp\kern-5pt\perp t_1 \} $$

The belief network for the distribution $(t_1,t_2,y_1y_2,h)$ is (p 52):

enter image description here

Now, by what I understand, when we are talking about the distribution $(t_1,t_2,y_1,y_2)$, we marginalize $h$ and the belief network becomes:

enter image description here

Question 1: So the edge connecting $y_1$ and $y_2$ is bidirectional, (right?) And so this means that $y_1 \perp\kern-5pt\perp t_2 | t_1$ because $t_1$ is not a collider for $y_1$ and $t_2$ (similar reasoning for the other independence statements). But we have that $y_1 \perp\kern-5pt\perp t_2 | y_2$ is not true because $y_2$ is a collider, right?

Question 2: The example in the book says: Consider the graph of the BN $$p(y_2|y_1,t_1,t_2)p(y_1|t_1)p(t_1)p(t_2)$$ For this we have $\mathcal{L}_G = \{y_1 \perp\kern-5pt\perp t_2 | t_1, t_2 \perp\kern-5pt\perp t_2\}$. Now, I don't understand why the BN is this:enter image description here This is a different graph from what I thought (the one above). How did they come up with this BN?

$\endgroup$
1
  • 1
    $\begingroup$ I think your interpretation is reasonable, but technically I don't see how your first equation (and your belief network) follows from the given equation with the $\sum_h$. The author seems to be trying to treat $h$ in a different way from the other variables. $\endgroup$
    – Karl
    Jul 6, 2022 at 16:41

1 Answer 1

1
+50
$\begingroup$

Your Question 1:
I completely agree and the explanation is correct. One single point: I don't know about how Barber is doing it, but often the edge between $y_1$ and $y_2$, which you called "bidirectional", is depicted as $\leftrightarrow$. Thus it becomes "visible" that e.g. $y_2$ is a collider on the path from $y_1$ to $t_2$. (This can actually be taken further to arrive at a special kind of graph, called MAG (Maximal Ancestral Graph), which then has the nice property that marginalizations of perfect maps stay perfect.)

Your Question 2:
I think Barber just wants to say that this Bayesian network (4.5.9), which you haw correctly drawn, is not a perfect map for the distribution (4.5.7) and is only an I-map. He goes on to say that there actually exists neither a BN nor an MN that would be a perfect map for the distribution (4.5.7).

Why has he picked this BN, in particular, to check whether it could be a perfect map? He doesn't say. Maybe, his reasoning is somehow like this: If $\;t_1 \perp\kern-5pt\perp\kern-8pt/\;\; y_2 \;|\; y_1$ then there should be an edge between $t_1$ and $y_2$ to ensure this conditional dependency. And he then points out that this attempt to find a perfect graph just doesn't work.

For what it's worth, his set $\mathcal L_P$ in (4.5.8) is, contrary to his assertion, not the list of all independence statements, e.g. $y_1 \perp\kern-5pt\perp t_2$ (i.e. unconditionally) is missing (at least in my copy of the book).

$\endgroup$
1
  • $\begingroup$ Hey Frank, thank you very much. I was wondering whether you'd be interested for a one-off gig. I'd basically like to hire you only to check my solutions to the past papers (3 papers) of a module which is based on Barber's Book, since I don't have solutions to these. Let me know if you're interested: valentinrastko[at]gmail.com $\endgroup$
    – user
    Jul 27, 2022 at 9:57

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .