58
$\begingroup$

I have been using Sebastian Thrun's course on AI and I have encountered a slightly difficult problem with probability theory.

He poses the following statement:

$$ P(R \mid H,S) = \frac{P(H \mid R,S) \; P(R \mid S)}{P(H \mid S)} $$

I understand he used Bayes' Rule to get the RHS equation, but fail to see how he did this. If somebody could provide a breakdown of the application of the rule in this problem that would be great.

$\endgroup$
1
  • $\begingroup$ When you have a random variable like $S$ that is to the right of the conditioning bar in every expression, you may mentally eliminate it, bearing in mind that all probabilities are conditioned on $S$. When you do this, the statement is simply Bayes' Rule. $\endgroup$
    – Jim Ferry
    Aug 24, 2023 at 20:13

4 Answers 4

80
$\begingroup$

Taking it one step at a time: $$\begin{align} \mathsf P(R\mid H, S) & = \frac{\mathsf P(R,H,S)}{\mathsf P(H, S)} \\[1ex] & =\frac{\mathsf P(H\mid R,S)\,\mathsf P(R, S)}{\mathsf P(H, S)} \\[1ex] & =\frac{\mathsf P(H\mid R,S)\,\mathsf P(R\mid S)\,\mathsf P(S)}{\mathsf P(H, S)} \\[1ex] & =\frac{\mathsf P(H\mid R,S)\,\mathsf P(R\mid S)}{\mathsf P(H\mid S)}\frac{\mathsf P(S)}{\mathsf P(S)} \\[1ex] & =\frac{\mathsf P(H\mid R,S)\;\mathsf P(R\mid S)}{\mathsf P(H\mid S)} \end{align}$$

$\endgroup$
7
  • $\begingroup$ In the first equality the definition of conditional probability is used. In the numerator of the second line the product rule (chain rule) is used as well as in the denominator to derive the fourth one. $\endgroup$
    – Arraval
    Dec 16, 2019 at 16:05
  • $\begingroup$ @Arraval It is the definition conditional probability in every step, save for the last, where we cancel common factors. $\endgroup$ Dec 16, 2019 at 20:30
  • $\begingroup$ You are right. Maybe there is a bit of ambiguity and both concepts are the same? Or I'm completely confused (most likely) I read these two articles from wikipedia: Chain rule: en.wikipedia.org/wiki/Chain_rule_%28probability%29 Conditional probability: en.wikipedia.org/wiki/Conditional_probability $\endgroup$
    – Arraval
    Dec 17, 2019 at 11:16
  • 1
    $\begingroup$ Hello @GrahamKemp can you tell me if the comma in $P(R | H, S)$ means $P(R | H ∩ S)$ please? $\endgroup$ Dec 26, 2021 at 16:27
  • 2
    $\begingroup$ Yes. Lists in probability functions are conjunctive. @GennaroArguzzi $\endgroup$ Dec 26, 2021 at 18:01
18
$\begingroup$

You don't really need Bayes' Theorem. Just apply the definition of conditional probability in two ways. Firstly,

\begin{eqnarray*} P(R\mid H,S) &=& \dfrac{P(R,H\mid S)}{P(H\mid S)} \\ && \\ \therefore\quad P(R,H\mid S) &=& P(R\mid H,S)P(H\mid S). \end{eqnarray*}

Secondly,

\begin{eqnarray*} P(H\mid R,S) &=& \dfrac{P(R,H\mid S)}{P(R\mid S)} \\ && \\ \therefore\quad P(R,H\mid S) &=& P(H\mid R,S)P(R\mid S). \end{eqnarray*}

Combine these two to get the result.

$\endgroup$
5
  • 2
    $\begingroup$ I would vote up, but alas, I do not have enough reputation to vote up an answer to my own question :/ $\endgroup$
    – slyaer
    May 14, 2015 at 9:55
  • 1
    $\begingroup$ How to you get from $P(R|H,S)$ to $P(R,H|S)/P(H|S)$? $\endgroup$
    – Code Pope
    Jul 19, 2019 at 16:35
  • $\begingroup$ @MoProg From the definition of conditional probability. Just as you can have $P(A\mid B) = P(A,B)/P(B)$, which is from the standard definition, you can also have $P(A\mid B,C) = P(A,B\mid C)/P(B\mid C)$, for events $A,B,C$. $\endgroup$
    – Mick A
    Jul 19, 2019 at 22:50
  • 1
    $\begingroup$ Sorry @MickA, but I still don't understand how you use the definition of conditional probablity $P(X|Y)= P(X,Y)/P(Y)$ to get from $P(A|B,C)$ to $P(A,B|C)/P(B|C)$. If I use the definition of conditional probability I would have $P(A|B,C) = P(A,B,C)/P(B,C)$ , because I will substitute $Y = B,C$. $\endgroup$
    – Code Pope
    Jul 22, 2019 at 7:57
  • 3
    $\begingroup$ @MoProg We can derive it this way: $\frac{P(A,B|C)}{P(B|C)} = \frac{P(A,B,C)/P(C)}{P(B,C)/P(C)}$ $ = \frac{P(A,B,C)}{P(B,C)} =P(A|B,C)$. $\endgroup$
    – Mick A
    Jul 22, 2019 at 13:59
1
$\begingroup$

enter image description here

Hopefully, it will help to understand the above derivation.

$\endgroup$
0
$\begingroup$

Based on Graham Kemp's answer. $$Pr(R|(H,S)) = \frac{Pr(R,H,S)}{Pr(H,S)} \tag{1}$$ And for $$Pr(H|(R,S)) = \frac{Pr(R,H,S)}{Pr(R,S)}\tag{2.1}$$ $$Pr(R,H,S) = Pr(H|(R,S))Pr(R,S) \tag{2.2}$$ Substitute 2.1 to 1, we got $$Pr(R|(H,S)) = \frac{Pr(H|(R,S))Pr(R,S)}{Pr(H,S)} \tag{3}$$ We done the same for $Pr(R,S)$ and $Pr(H,S)$ $$Pr(R,S) = Pr(R|S)P(S) \tag{4}$$ $$Pr(H,S) = Pr(H|S)P(S) \tag{5}$$ Substitute 4,5 to 1, we got $$Pr(R|(H,S)) = \frac{Pr(H|(R,S))Pr(R|S)P(S)}{Pr(H|S)P(S)} \tag{6.1}$$ The term $H(s)$ is cancel-out, so now we got $$Pr(R|(H,S)) = \frac{Pr(H|(R,S))Pr(R|S)}{Pr(H|S)} \tag{6.2}$$

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .