33
$\begingroup$

I have been using Sebastian Thrun's course on AI and I have encountered a slightly difficult problem with probability theory.

He poses the following statement:

$$ P(R \mid H,S) = \frac{P(H \mid R,S) \; P(R \mid S)}{P(H \mid S)} $$

I understand he used Bayes' Rule to get the RHS equation, but fail to see how he did this. If somebody could provide a breakdown of the application of the rule in this problem that would be great.

$\endgroup$
45
$\begingroup$

Taking it one step at a time: $$\begin{align} \mathsf P(R\mid H, S) & = \frac{\mathsf P(R,H,S)}{\mathsf P(H, S)} \\[1ex] & =\frac{\mathsf P(H\mid R,S)\,\mathsf P(R, S)}{\mathsf P(H, S)} \\[1ex] & =\frac{\mathsf P(H\mid R,S)\,\mathsf P(R\mid S)\,\mathsf P(S)}{\mathsf P(H, S)} \\[1ex] & =\frac{\mathsf P(H\mid R,S)\,\mathsf P(R\mid S)}{\mathsf P(H\mid S)}\frac{\mathsf P(S)}{\mathsf P(S)} \\[1ex] & =\frac{\mathsf P(H\mid R,S)\;\mathsf P(R\mid S)}{\mathsf P(H\mid S)} \end{align}$$

$\endgroup$
4
  • $\begingroup$ In the first equality the definition of conditional probability is used. In the numerator of the second line the product rule (chain rule) is used as well as in the denominator to derive the fourth one. $\endgroup$ – Arraval Dec 16 '19 at 16:05
  • $\begingroup$ @Arraval It is the definition conditional probability in every step, save for the last, where we cancel common factors. $\endgroup$ – Graham Kemp Dec 16 '19 at 20:30
  • $\begingroup$ You are right. Maybe there is a bit of ambiguity and both concepts are the same? Or I'm completely confused (most likely) I read these two articles from wikipedia: Chain rule: en.wikipedia.org/wiki/Chain_rule_%28probability%29 Conditional probability: en.wikipedia.org/wiki/Conditional_probability $\endgroup$ – Arraval Dec 17 '19 at 11:16
  • $\begingroup$ According to the Chain rule wikipedia article: "...the chain rule permits the calculation of any member of the joint distribution of a set of random variables using only conditional probabilities" that's why I thought you used the Chain Rule in the numerator of the RHS in the first line because you express a joint probability P(R,H,S) in terms of a conditional P(H|R,S)P(R,S) just like P(A,B)=P(B|A)P(A) so I thought A=R,S and B=H. As I said before, am I completely lost or there is some ambiguity in both concepts. $\endgroup$ – Arraval Dec 17 '19 at 11:27
14
$\begingroup$

You don't really need Bayes' Theorem. Just apply the definition of conditional probability in two ways. Firstly,

\begin{eqnarray*} P(R\mid H,S) &=& \dfrac{P(R,H\mid S)}{P(H\mid S)} \\ && \\ \therefore\quad P(R,H\mid S) &=& P(R\mid H,S)P(H\mid S). \end{eqnarray*}

Secondly,

\begin{eqnarray*} P(H\mid R,S) &=& \dfrac{P(R,H\mid S)}{P(R\mid S)} \\ && \\ \therefore\quad P(R,H\mid S) &=& P(H\mid R,S)P(R\mid S). \end{eqnarray*}

Combine these two to get the result.

$\endgroup$
5
  • 2
    $\begingroup$ I would vote up, but alas, I do not have enough reputation to vote up an answer to my own question :/ $\endgroup$ – slyaer May 14 '15 at 9:55
  • 1
    $\begingroup$ How to you get from $P(R|H,S)$ to $P(R,H|S)/P(H|S)$? $\endgroup$ – Code Pope Jul 19 '19 at 16:35
  • $\begingroup$ @MoProg From the definition of conditional probability. Just as you can have $P(A\mid B) = P(A,B)/P(B)$, which is from the standard definition, you can also have $P(A\mid B,C) = P(A,B\mid C)/P(B\mid C)$, for events $A,B,C$. $\endgroup$ – Mick A Jul 19 '19 at 22:50
  • 1
    $\begingroup$ Sorry @MickA, but I still don't understand how you use the definition of conditional probablity $P(X|Y)= P(X,Y)/P(Y)$ to get from $P(A|B,C)$ to $P(A,B|C)/P(B|C)$. If I use the definition of conditional probability I would have $P(A|B,C) = P(A,B,C)/P(B,C)$ , because I will substitute $Y = B,C$. $\endgroup$ – Code Pope Jul 22 '19 at 7:57
  • 3
    $\begingroup$ @MoProg We can derive it this way: $\frac{P(A,B|C)}{P(B|C)} = \frac{P(A,B,C)/P(C)}{P(B,C)/P(C)}$ $ = \frac{P(A,B,C)}{P(B,C)} =P(A|B,C)$. $\endgroup$ – Mick A Jul 22 '19 at 13:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.