2
$\begingroup$

I'm having some difficulty understanding Bayes' theorem with multiple events. I'm trying to put together a Bayesian network. I have four independent probabilities but I have found that A, B and C can affect the probability of D, so my question is: how do I write the formula for $p(D | B, C, A)$?

As an example, suppose I was given: p(A) = 31.6%, p(B) = 71%, p(C) = 4.5%, and p(D) = 22%, how would I get $p(D|B,C,A)$?

$\endgroup$
  • $\begingroup$ -1: People are volunteering their time to help. Please take the time to use punctuation to seperate sentences and read it over before posting. I've attempted to clean it up. $\endgroup$ – Garrett Nov 24 '14 at 22:59
  • $\begingroup$ apologizes i do not want my bad grammar skills to make me seem ungrateful or rude to anyone. $\endgroup$ – user3476732 Nov 24 '14 at 23:17
  • $\begingroup$ No problem, and it's an interesting question by the way! Just make sure to put periods and question marks between sentences and capitalize "I" and the beginning of sentences. Check out this style guide. $\endgroup$ – Garrett Nov 24 '14 at 23:22
  • $\begingroup$ yes i do. By the time i come to this site to ask a question I've usually exhausted myself, then lots of mistakes happen. $\endgroup$ – user3476732 Nov 24 '14 at 23:36
  • $\begingroup$ What do you mean when you say that D may be "void"? $\endgroup$ – Garrett Nov 24 '14 at 23:36
1
$\begingroup$

We can look to Bayes formula for inspiration. It can be derived from the definition of the joint distribution:

$$P(A, B) = P(A|B) \,P(B) = P(B|A)\,P(A)$$ and rearraning to give $$P(B|A) = \frac{P(A|B) \,\,P(B)}{P(A)}$$

For the case of 4 variables, we have many more options. Below is one example of a formula

Example

We could write the joint distribution as:

\begin{align}P(A,B,C,D) =& P(A | B,C,D) \, P(B,C,D) \\ =& P(B | A,C,D) \, P(A,C,D) \\ =& P(C | A,B,D) \, P(A,B,D) \\ =& P(D | A,B,C) \, P(A,B,C) \end{align} To see where this came from, in the first line, I'm treating the eventrandom variables $(B,C,D)$ as a single random variable, so it's similar to just writing $P(A,Y)=P(A|Y)\,P(Y)$ where $Y = (B,C,D)$ is a 3-dimensional random variable.

Therefore, one option is to rearrange the above to get

$$P(D|A,B,C) = \frac{ P(A | B,C,D) \, P(B,C,D) } {P(A,B,C)} \tag{1}$$

If you happy with this you can stop here.

Example (cont'd)

In the above formula, you may notice that $P(B,C,D)$ and $P(A,B,C)$ can be further broken down if desired. Similar to Example 1, we could do \begin{align}P(B,C,D) =& P(B|C,D) \, P(C,D) \\ =& P(C|B,D) \, P(B,D) \\ =& P(D|B,C) \, P(B,C) \end{align} I'm going to arbitrarily choose the first line of the above equations, but notice that $P(C,D)$ can be further broken down to give $$P(B,C,D) = P(B|C,D) \, P(C|D) \, P(D)$$

In a similar fashion, we could choose to express $P(A,B,C)$ as $$P(A,B,C) = P(A|B,C) \,P(B|C) \,P(C)$$ (in the context of probability, the above equation is called the Chain Rule)

Substituting these back into Eq (1) yields

$$P(D|A,B,C) = \frac{P(A|B,C,D)\,P(B|C,D)\,P(C|D)\,P(D)} {P(A|B,C)\, P(B|C)\, P(C) }$$

Conclusion

I made a lot of arbitrary choices in how I expressed the joint distributions above. Therefore there are many different formulas you could have ended up at.

Note that just like in the normal Bayes' theorem where we need more than $P(A)$ and $P(B)$ to get $P(B|A)$, here, we also will need more than just $P(A)$, $P(B)$, $P(C)$ and $P(D)$ to get $P(D|A,B,C)$.

$\endgroup$
  • $\begingroup$ This confuses events and random variables. The question being about the former, mentions of joint distributions or tuples of random variables should be replaced by intersections of events. For example, $(B,C,D)$ is simply irrelevant and the formula $$P(A,B,C,D)=P(A\mid B,C,D)P(B,C,D)$$ actually means $$P(A\cap B\cap C\cap D)=P(A\mid B\cap C\cap D)P(B\cap C\cap D).$$ On the other hand, the last sentence of the post is spot on. $\endgroup$ – Did Mar 26 '16 at 9:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.