Zero probability condition: Thinking about '$P(H)=\int_{b \in \mathbb R}P(H|B=b)f_B(b)$' Let $B$ be a continuous random variable. Let $H$ be an event.

*

*Am I right to think it does not necessarily make sense to say '$P(H)=\int_{b \in \mathbb R}P(H|B=b)f_B(b)$'?

My guess: Well based on Wiki, I think yes, i.e. we cannot do this for just any $H$ because $P(H|B=b)$ need not be defined. Furthermore, even if we somehow define $P(H|B=b)$, I think we'll still have to think about defining the integral $\int_{b \in \mathbb R}P(H|B=b)f_B(b)$, depending on the definition of $P(H|B=b)$.


*Is '$P(H|B=b)$' well-defined if $H=\{Y \in U\}$ for any continuous random variable $Y$ s.t. $Y$ and $B$ have a continuous joint pdf and for any interval $U$? (I forgot if any 2 continuous random variables necessarily have a well-defined joint pdf. Also, I'm trying not think of $U$ as an arbitrary Borel set.)


*With the same conditions as in (2) and assuming the answer to (2) is affirmative, does it necessarily make sense to say $P(H)=\int_{b \in \mathbb R}P(H|B=b)f_B(b)$, and is such equation correct?
My guess: I believe it makes sense and then is correct: Pretend $U=(1,7)$. Then: $LHS = P(H)=\int_1^7 f_Y(y) dy$. $RHS= \int_{b \in \mathbb R}P(Y \in (1,7)|B=b)f_B(b) db = \int_{b \in \mathbb R} \int_1^7 f_{Y|B=b}(y) f_B(b) dy db = \int_{b \in \mathbb R} \int_1^7 f_{Y,B}(y,b) dy db$
Then pretend I know what Fubini's theorem is to get
$=  \int_1^7 \int_{b \in \mathbb R} f_{Y,B}(y,b)  db dy$
$=  \int_1^7 f_{Y}(y) dy$


*Re Fubini's theorem, is there a way to argue this at an elementary probability level?

 A: This is essentially the same as the "law of iterated expectations"
$$ E[1_H] = E[E[1_H|B]]$$
where $H$ is an event and $1_H$ is an indicator function:
$$ 1_H = \left\{\begin{array}{cc}
1 & \mbox{ if $H$ occurs} \\
0 & \mbox{ else} \end{array}\right.$$
So you can define $P[H|B=b]$ as a function $g(b)$, where $g$ is a measurable function such that $g(B)$ gives a version of the conditional expectation $E[1_H|B]$.

A non-measure-theoretic way to define $P[H|B=b]$ is:
$$ P[H|B=b] = \left\{\begin{array}{cc}
\frac{f_{B|H}(b)P[H]}{f_B(b)} & \mbox{ if $P[H]>0$}\\
0 & \mbox{if $P[H]=0$} \end{array}\right.$$
which also requires $B$ to have a PDF $f_B(x)$ that satisfies $f_B(b)>0$, and requires the existence of the PDF $f_{B|H}(x)$.
You can see how this definition satisfies
$$ P[H] = \int_{-\infty}^{\infty} P[H|B=b]f_B(b)db$$
You might critique the right-hand-side of the above equation by noticing $P[H|B=b]$ is not defined when $f_B(b)=0$, indeed the right-hand-side should more precisely integrate only over points where $f_B(b)>0$. This is similar to the classical statement of the law of total probability for an event $A$ and a partition of the sample space $\{B_i\}_{i=1}^{\infty}$
$$P[A] = \sum_{i=1}^{\infty} P[A|B_i]P[B_i]$$
It is implicitly understood that the right-hand-side really only sums over those indices $i$ for which $P[B_i]>0$.  It could more precisely be written
$$ P[A] = \sum_{i=1}^{\infty} P[A \cap B_i] = \sum_{i\in \mathbb{N}:P[B_i]>0} P[A|B_i]P[B_i]$$

