43
$\begingroup$

I am wondering how I would apply Bayes rule to expand an expression with multiple variables on either side of the conditioning bar.

In another forum post, for example, I read that you could expand $P(a,z \mid b)$ using Bayes rule like this (see Summing over conditional probabilities): $$P( a,z \mid b) = P(a \mid z,b) P(z \mid b)$$

However, directly using Bayes rule to expand $P(a,z \mid b)$ doesn't seem to be the right way to start out:

$$P(a,z\mid b) = { P(b\mid a,z)P(a,z) \over P(a)}$$

$\endgroup$
3
  • 1
    $\begingroup$ The first equation is not Bayes' rule, it's just the definition of conditional probability. You don't need Bayes' rule at all, in fact, here. In your second equation, $P(a)$ should be $P(b)$ in the denominator. $\endgroup$
    – Kirill
    Jun 1, 2013 at 22:16
  • $\begingroup$ aiqus.com/forum/questions/5627/… $\endgroup$
    – Ankit
    Feb 10, 2017 at 12:05
  • $\begingroup$ For the Bayes rule with two conditions P(A| B,C), check math.stackexchange.com/questions/2463896/… $\endgroup$ Nov 5, 2020 at 15:10

3 Answers 3

55
$\begingroup$

Note that you didn't apply Bayes' Rule correctly; Bayes' Rule says that:

$$P(X|Y)={P(Y|X)P(X) \over P(Y)}$$

so your denominator should have actually been $P(b)$. Instead, I will use the definition of conditional probability and multiplication rule (which together imply Bayes' Rule):

\begin{array}{cc} P(X|Y) =\dfrac{P(X,Y)}{P(Y)} & (1)\\ P(X)P(Y|X) =P(X,Y)=P(Y)P(X|Y) & (2) \end{array}

Thus, observe that: $$ \begin{array}{r@{}ll} P\big( (a,z) \mid b \big) &= \dfrac{P(a,z,b)}{P(b)} & \text{by (1), where } X=a,z \text{ and } Y=b\\ &= \dfrac{P(z,b)P\big(a \mid (z,b) \big)}{P(b)} &\text{by (2), where } X=a \text{ and } Y=z,b\\ &= \dfrac{P(b)P(z \mid b)P\big(a \mid (z,b) \big)}{P(b)} &\text{by (2), where } X=z \text{ and } Y=b\\ &= P(z \mid b)P\big(a \mid (z,b) \big) \\ &= P\big(a \mid (z,b) \big) P(z \mid b) \\ \end{array} $$ as desired.

$\endgroup$
8
$\begingroup$

FYI, using the chain rule, P(a,z,b) = P(a|z,b)*P(z|b)*P(b)

$\endgroup$
2
  • 1
    $\begingroup$ It should be P(a,z|b) $\endgroup$
    – Babai
    Nov 26, 2014 at 9:58
  • $\begingroup$ Plus, $P(a,z,b)=P(a,z|b) P(b)\quad\equiv \quad P(a|z,b)P(z,b)$. $\endgroup$
    – nyxee
    Feb 21, 2017 at 2:42
2
$\begingroup$

Since the op mentioned expand an expression with multiple variables on EITHER side of the conditioning bar., here is my attempt to derive P(a|z,b) using Bayesian rule, assuming that A is a discrete variable:

$$\begin{align} P(A=a1|z,b) &= \frac{P(a_1, z, b)}{P(z,b)} \\ &= \frac{P(a_1,b|z) P(z)}{P(b|z)P(z)} \\ &= \frac{P(a_1,b|z)}{P(b|z)} \\ &= \frac{P(b|a_1,z)P(a_1|z)}{P(b|z)} \\ &= \frac{P(b|a_1,z)P(a_1|z)}{P(b|a_1,z)P(a_1|z) + P(b|a_2,z)P(a_2|z) + ...} \end{align}$$

Compare this with the "single variable" version where

$$\begin{align} P(A=a_1|b) &= \frac{P(a_1,b)}{P(b)} \\ &= \frac{P(b|a_1)P(a_1)}{P(b)} \\ &= \frac{P(b|a_1)P(a_1)} {P(b|a_1)P(a_1) + P(b|a_2)P(a_2) + ..} \end{align}$$

Can we think of Z as a extra "condition" that's added to both the numerator and denominator?

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .