I am doing problems related to Bayesian network. After reading the theory part I am able to understand that by making a network or reducing a problem to some Bayesian network, we are simplifying a process for computing the joint probability of events, by minimizing the number of parameters for calculating the joint probabilities. But I have one doubt. Assume that we have obtained a formula for computing the joint probabilities from the network. Can I compute probabilities from this formula for each combination of events or can I do that only after making the table for joint distribution and then finding marginal or conditional probabilities encountered in my problem. I mean that suppose the following is the formula we obtained from the network:

$\hspace{10ex}P(D,S,G,I,L)=P(D)\cdot P(I)\cdot P(G\mid(ID))\cdot P(L\mid G)\cdot P(S\mid I)$

And all variables are binary. Now we have tables for each of the probabilities that appear in the above formula. But now I want to know the probability for $P(L=0\mid (S=0,I=1))$ from the existing probabilities. What is the process for doing that?


1 Answer 1


A comment first: When constructing a BN you are not necessarily interested in minimizing the number of nodes and/or edges. Basically, a BN is a way of encoding a joint distribution on some parameters. There are many different BN's that will encode the very same joint distribution and which one you choose depends on many factors. Computationally, you wish your BN to be small so as to allow easier inference on it. But when constructing a BN (either by utilizing expert knowledge or by machine learning techniques) you very often don't want your BN to be too small (compared to the available statistical date) since that is prone to over-fitting. Generally, the larger the BN the more expressive power it has but the more computationally difficult it is to perform inference. Choosing BN's for a particular task is more of an art than anything else.

As for your question, if what you are asking is: given the joint distribution obtained from the full CPD's at each notd can I compute all sorts of conditional probabilities? The answer is yes, since if you have the CPD's then you have essentially all of the joint distributions and from these you can compute conditional probabilities. This really has nothing to do with any BN. It's just that given random variables, if you know the joint distributions then you can compute any relevant statistical quantity. I hope this helps.

  • $\begingroup$ Ok..but isn't it one of the motivation is minimizing the number of parameters. For example, the illustration which I gave in the above post if solved without using BN we have to compute: P(D,S,G,I,L)=p(D)*P(S/D)*P(G/SD)*P(I/DSG)*P(L/DSGL) and in order to computing this we will require: 1+2+4+8+16=31 parameters(conditional probabilities) and with the model it will require only:10 (conditional probabilities) $\endgroup$
    – Manish
    Nov 22, 2012 at 1:08
  • $\begingroup$ The BN is just a combinatorial structure that supports the CPD's and the distributions. If you have the individual CPD's then you can compute anything you want from them with or without the underlying BN. The point (or rather a point) of BN's is that they can be learned from statistical data and then you really don't want to minimize the size of the BN too much. Also when designing BN's by hand you might find that adding latent or hidden variables actually simplifies things since the CPD's may become easier to compute. $\endgroup$ Nov 22, 2012 at 2:00
  • $\begingroup$ Why BN being too small is prone to over-fitting? This seems counter-intuitive. I would assume that over-fitting is possible when there is too little available statistical data or when there are too many nodes in BN. $\endgroup$
    – Georgy
    Mar 7, 2018 at 14:12
  • $\begingroup$ @Georgy if you think of over-fitting as spending so much time with the training data that you start recognising particularities of the training data that isn't really characteristic of the generic phenomenon at hand, then it should become clearer. When the BN is small its capacity for nuances is lesser than a larger one. Therefore, it will more quickly become over-fit on data. $\endgroup$ Mar 8, 2018 at 10:52

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .