I came across this expression in the Intro to Probability book I am studying:


Could anyone please explain how is this obtained. From a simple application of Bayes Rule, shouldn't it be:

$P(A,B|C)=\frac{P(C|A,B)P(A,B)}{P(C)}$ where $P(A,B) = P(A|B)P(B)$ ?

  • $\begingroup$ Is $P(A,B|C)=\frac{P(C)P(B|C)P(A|B,C)}{P(C)}$ correct? As the $P(C)$ just cancels. $\endgroup$ – BLAZE Dec 23 '15 at 1:52
  • 1
    $\begingroup$ Yep, it is. They cancel out the $P(C)$ and write $P(B|C)P(A|B,C)$ as the final expression. $\endgroup$ – QPTR Dec 23 '15 at 2:17

By the definition of conditional probability: $P(Y \mid X)=P(X,Y)/P(X) \implies P(X,Y)=P(Y\mid X) P(X)$. Hence (think of $A,B$ as a single multivariate variable):

$$P(A,B\mid C)=\frac{P(A,B,C)}{P(C)}$$

Then, by the chain rule of probability, $P(A,B,C)=P(A\mid B,C)P(B,C)$ and $P(B,C)=P(B\mid C)P(C)$

  • $\begingroup$ Thanks for the answer. Shouldn't ${P(Y|X) = P(X,Y)|P(X)}$ according to Bayes' rule? I do understand now how the answer was obtained, so its just a different way of interpreting the $P(A,B,C)$? The way how $P(A,B,C)$ was split into $P(A|B,C)P(B,C)$ instead of $P(C|A,B)P(A,B)$. Are both of them equivalent? $\endgroup$ – QPTR Dec 23 '15 at 2:10
  • $\begingroup$ Error fixed. Yes, both forms are equivalent $\endgroup$ – leonbloy Dec 23 '15 at 2:19
  • $\begingroup$ Note that this has nothing to do with Bayes' rule $\endgroup$ – ovolve Dec 23 '15 at 2:22
  • 1
    $\begingroup$ @QPTR Bayes' rule says $P(A,B|C)=\frac{P(C|A,B)P(A,B)}{P(C)}$. It looks close to what you are given, but the difference is in your problem $C$ is never conditioned on $A,B$. You can get your derivation with just the definition of conditional probability and the chain rule. Flipping things with Bayes doesn't help $\endgroup$ – ovolve Dec 23 '15 at 2:33
  • 1
    $\begingroup$ @QTPR Yes, you can choose what you condition on, and in practice it will be obvious how you should do it because you'll want to condition on the thing that you know! $\endgroup$ – ovolve Dec 23 '15 at 2:35

The extension of Baye's rule is such that $$P(A,B|C)=\frac{\color{blue}{P(A)}\color{blue}{P(B|A)}P(C|A,B)}{\color{blue}{P(B)}P(C|B)}\tag{1}$$

The formula can be seen as an extension of $$P(A|B)=\frac{\color{blue}{P(A)P(B|A)}}{\color{blue}{P(B)}}\tag{2}$$ This is not a derivation of course; I'm just trying to show you that the $\color{blue}{\mathrm{blue}}$ terms are common to both $(1)$ and $(2)$ and if you reverse the conditioning for the extra variable ($C$) by placing the conditional probabilities in both numerator and denominator you arrive at $(1)$.

  • 1
    $\begingroup$ Thanks for the answer! I don't understand $(1)$. From my understanding it seems like there should only be $P(C)$ in the denominator. How was $P(B)P(C|B)$ obtained? $\endgroup$ – QPTR Dec 23 '15 at 2:16
  • $\begingroup$ @QPTR See this link under section $3.3$ for more info, hope this helps. $\endgroup$ – BLAZE Dec 23 '15 at 2:19
  • $\begingroup$ @QPTR You're absolutely right, my mistake sorry it was a typo. $\endgroup$ – BLAZE Dec 23 '15 at 2:23
  • $\begingroup$ @QPTR I'm curious; Do you have a source by any chance for your formula in your question? As I have never seen it written like that before. Thanks. $\endgroup$ – BLAZE Dec 23 '15 at 2:36
  • 1
    $\begingroup$ Yeah, its this one, Intro to Prob by Bertsekas and Tsitsiklis. Here's the pdf. Its under the Conditional Independence heading on Page 36. $\endgroup$ – QPTR Dec 23 '15 at 2:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.