I am wondering how I would apply Bayes rule to expand an expression with multiple variables on either side of the conditioning bar.

In another forum post, for example, I read that you could expand $P(a,z \mid b)$ using Bayes rule like this (see Summing over conditional probabilities): $$P( a,z \mid b) = P(a \mid z,b) P(z \mid b)$$

However, directly using Bayes rule to expand $P(a,z \mid b)$ doesn't seem to be the right way to start out:

$$P(a,z\mid b) = { P(b\mid a,z)P(a,z) \over P(a)}$$

  • $\begingroup$ The first equation is not Bayes' rule, it's just the definition of conditional probability. You don't need Bayes' rule at all, in fact, here. In your second equation, $P(a)$ should be $P(b)$ in the denominator. $\endgroup$ – Kirill Jun 1 '13 at 22:16
  • $\begingroup$ aiqus.com/forum/questions/5627/… $\endgroup$ – Ankit Feb 10 '17 at 12:05

Note that you didn't apply Bayes' Rule correctly; Bayes' Rule says that:

$$P(X|Y)={P(Y|X)P(X) \over P(Y)}$$

so your denominator should have actually been $P(b)$. Instead, I will use the definition of conditional probability and multiplication rule (which together imply Bayes' Rule):

\begin{array}{cc} P(X|Y) =\dfrac{P(X,Y)}{P(Y)} & (1)\\ P(X)P(Y|X) =P(X,Y)=P(Y)P(X|Y) & (2) \end{array}

Thus, observe that: $$ \begin{array}{r@{}ll} P( a,z \mid b) &= \dfrac{P(a,z,b)}{P(b)} & \text{by (1), where } X=a,z \text{ and } Y=b\\ &= \dfrac{P(z,b)P(a \mid z,b)}{P(b)} &\text{by (2), where } X=a \text{ and } Y=z,b\\ &= \dfrac{P(b)P(z \mid b)P(a \mid z,b)}{P(b)} &\text{by (2), where } X=z \text{ and } Y=b\\ &= P(z \mid b)P(a \mid z,b) \\ &= P(a \mid z,b) P(z \mid b) \\ \end{array} $$ as desired.


FYI, using the chain rule, P(a,z,b) = P(a|z,b)*P(z|b)*P(b)

  • 1
    $\begingroup$ It should be P(a,z|b) $\endgroup$ – Babai Nov 26 '14 at 9:58
  • $\begingroup$ Plus, $P(a,z,b)=P(a,z|b) P(b)\quad\equiv \quad P(a|z,b)P(z,b)$. $\endgroup$ – nyxee Feb 21 '17 at 2:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.