45
$\begingroup$

I have been using Sebastian Thrun's course on AI and I have encountered a slightly difficult problem with probability theory.

He poses the following statement:

$$ P(R \mid H,S) = \frac{P(H \mid R,S) \; P(R \mid S)}{P(H \mid S)} $$

I understand he used Bayes' Rule to get the RHS equation, but fail to see how he did this. If somebody could provide a breakdown of the application of the rule in this problem that would be great.

$\endgroup$

2 Answers 2

Reset to default
64
$\begingroup$

Taking it one step at a time: $$\begin{align} \mathsf P(R\mid H, S) & = \frac{\mathsf P(R,H,S)}{\mathsf P(H, S)} \\[1ex] & =\frac{\mathsf P(H\mid R,S)\,\mathsf P(R, S)}{\mathsf P(H, S)} \\[1ex] & =\frac{\mathsf P(H\mid R,S)\,\mathsf P(R\mid S)\,\mathsf P(S)}{\mathsf P(H, S)} \\[1ex] & =\frac{\mathsf P(H\mid R,S)\,\mathsf P(R\mid S)}{\mathsf P(H\mid S)}\frac{\mathsf P(S)}{\mathsf P(S)} \\[1ex] & =\frac{\mathsf P(H\mid R,S)\;\mathsf P(R\mid S)}{\mathsf P(H\mid S)} \end{align}$$

$\endgroup$
6
  • $\begingroup$ In the first equality the definition of conditional probability is used. In the numerator of the second line the product rule (chain rule) is used as well as in the denominator to derive the fourth one. $\endgroup$
    – Arraval
    Dec 16, 2019 at 16:05
  • $\begingroup$ @Arraval It is the definition conditional probability in every step, save for the last, where we cancel common factors. $\endgroup$
    – Graham Kemp
    Dec 16, 2019 at 20:30
  • $\begingroup$ You are right. Maybe there is a bit of ambiguity and both concepts are the same? Or I'm completely confused (most likely) I read these two articles from wikipedia: Chain rule: en.wikipedia.org/wiki/Chain_rule_%28probability%29 Conditional probability: en.wikipedia.org/wiki/Conditional_probability $\endgroup$
    – Arraval
    Dec 17, 2019 at 11:16
  • 1
    $\begingroup$ Hello @GrahamKemp can you tell me if the comma in $P(R | H, S)$ means $P(R | H ∩ S)$ please? $\endgroup$
    – Gennaro Arguzzi
    Dec 26, 2021 at 16:27
  • 1
    $\begingroup$ Yes. Lists in probability functions are conjunctive. @GennaroArguzzi $\endgroup$
    – Graham Kemp
    Dec 26, 2021 at 18:01
17
$\begingroup$

You don't really need Bayes' Theorem. Just apply the definition of conditional probability in two ways. Firstly,

\begin{eqnarray*} P(R\mid H,S) &=& \dfrac{P(R,H\mid S)}{P(H\mid S)} \\ && \\ \therefore\quad P(R,H\mid S) &=& P(R\mid H,S)P(H\mid S). \end{eqnarray*}

Secondly,

\begin{eqnarray*} P(H\mid R,S) &=& \dfrac{P(R,H\mid S)}{P(R\mid S)} \\ && \\ \therefore\quad P(R,H\mid S) &=& P(H\mid R,S)P(R\mid S). \end{eqnarray*}

Combine these two to get the result.

$\endgroup$
5
  • 2
    $\begingroup$ I would vote up, but alas, I do not have enough reputation to vote up an answer to my own question :/ $\endgroup$
    – slyaer
    May 14, 2015 at 9:55
  • 1
    $\begingroup$ How to you get from $P(R|H,S)$ to $P(R,H|S)/P(H|S)$? $\endgroup$
    – Code Pope
    Jul 19, 2019 at 16:35
  • $\begingroup$ @MoProg From the definition of conditional probability. Just as you can have $P(A\mid B) = P(A,B)/P(B)$, which is from the standard definition, you can also have $P(A\mid B,C) = P(A,B\mid C)/P(B\mid C)$, for events $A,B,C$. $\endgroup$
    – Mick A
    Jul 19, 2019 at 22:50
  • 1
    $\begingroup$ Sorry @MickA, but I still don't understand how you use the definition of conditional probablity $P(X|Y)= P(X,Y)/P(Y)$ to get from $P(A|B,C)$ to $P(A,B|C)/P(B|C)$. If I use the definition of conditional probability I would have $P(A|B,C) = P(A,B,C)/P(B,C)$ , because I will substitute $Y = B,C$. $\endgroup$
    – Code Pope
    Jul 22, 2019 at 7:57
  • 3
    $\begingroup$ @MoProg We can derive it this way: $\frac{P(A,B|C)}{P(B|C)} = \frac{P(A,B,C)/P(C)}{P(B,C)/P(C)}$ $ = \frac{P(A,B,C)}{P(B,C)} =P(A|B,C)$. $\endgroup$
    – Mick A
    Jul 22, 2019 at 13:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.