I have been using Sebastian Thrun's course on AI and I have encountered a slightly difficult problem with probability theory.

He poses the following statement:

$$ P(R \mid H,S) = \frac{P(H \mid R,S) \; P(R \mid S)}{P(H \mid S)} $$

I understand he used Bayes' Rule to get the RHS equation, but fail to see how he did this. If somebody could provide a breakdown of the application of the rule in this problem that would be great.


Taking it one step at a time: $$\begin{align} \mathsf P(R\mid H, S) & = \frac{\mathsf P(R,H,S)}{\mathsf P(H, S)} \\[1ex] & =\frac{\mathsf P(H\mid R,S)\,\mathsf P(R, S)}{\mathsf P(H, S)} \\[1ex] & =\frac{\mathsf P(H\mid R,S)\,\mathsf P(R\mid S)\,\mathsf P(S)}{\mathsf P(H, S)} \\[1ex] & =\frac{\mathsf P(H\mid R,S)\,\mathsf P(R\mid S)}{\mathsf P(H\mid S)}\frac{\mathsf P(S)}{\mathsf P(S)} \\[1ex] & =\frac{\mathsf P(H\mid R,S)\;\mathsf P(R\mid S)}{\mathsf P(H\mid S)} \end{align}$$


You don't really need Bayes' Theorem. Just apply the definition of conditional probability in two ways. Firstly,

\begin{eqnarray*} P(R\mid H,S) &=& \dfrac{P(R,H\mid S)}{P(H\mid S)} \\ && \\ \therefore\quad P(R,H\mid S) &=& P(R\mid H,S)P(H\mid S). \end{eqnarray*}


\begin{eqnarray*} P(H\mid R,S) &=& \dfrac{P(R,H\mid S)}{P(R\mid S)} \\ && \\ \therefore\quad P(R,H\mid S) &=& P(H\mid R,S)P(R\mid S). \end{eqnarray*}

Combine these two to get the result.

  • 1
    $\begingroup$ I would vote up, but alas, I do not have enough reputation to vote up an answer to my own question :/ $\endgroup$ – slyaer May 14 '15 at 9:55
  • $\begingroup$ How to you get from $P(R|H,S)$ to $P(R,H|S)/P(H|S)$? $\endgroup$ – Mo Prog Jul 19 at 16:35
  • $\begingroup$ @MoProg From the definition of conditional probability. Just as you can have $P(A\mid B) = P(A,B)/P(B)$, which is from the standard definition, you can also have $P(A\mid B,C) = P(A,B\mid C)/P(B\mid C)$, for events $A,B,C$. $\endgroup$ – Mick A Jul 19 at 22:50
  • $\begingroup$ Sorry @MickA, but I still don't understand how you use the definition of conditional probablity $P(X|Y)= P(X,Y)/P(Y)$ to get from $P(A|B,C)$ to $P(A,B|C)/P(B|C)$. If I use the definition of conditional probability I would have $P(A|B,C) = P(A,B,C)/P(B,C)$ , because I will substitute $Y = B,C$. $\endgroup$ – Mo Prog Jul 22 at 7:57
  • $\begingroup$ @MoProg We can derive it this way: $\frac{P(A,B|C)}{P(B|C)} = \frac{P(A,B,C)/P(C)}{P(B,C)/P(C)}$ $ = \frac{P(A,B,C)}{P(B,C)} =P(A|B,C)$. $\endgroup$ – Mick A Jul 22 at 13:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.