# Definition of Conditional Probability by Measure Theory

I was reading a book on information theory and entropy by Robert Gray, when I saw the following definition of conditional probability:

Given a probability space $(\Omega,\mathcal{B}, P)$ and a sub-$\sigma$-field $\mathcal{G}$, for any event $H\in\mathcal{B}$ the conditional probability $m(H\text{ }|\text{ }\mathcal{G})$ is defined as any function , say $g$, which satisfies the two properties:

(1) $g$ is measurable with respect to $\mathcal{G}$

(2) $\displaystyle\int_{G}ghdP=m(G\bigcap{}H)$; all $G\in\mathcal{G}$

I am quite confused with this definition since it is very different from the definition through joint probability of events.

I understand what measurable function, sub-$\sigma$-field and probability space are, and I'm guessing that the author is trying to definie the measure $m$ through the measurable function $g$, but I don't quite understand what the second requirement is saying. Especially, what does that h in $\displaystyle\int_{G}ghdP$ refer to? it just jumped out of nowhere in the book, so I'm suspecting that it may have some conventional meaning?

I'd appreciate it a lot if someone can help. Thank you!!

-
I think the $h$ is a typo and should not be there at all. – Nate Eldredge Jun 6 '12 at 18:43
Umm .. perhaps you want $h=m(\mathcal{G})$ for normalization (if $h=1$, then $H=G=\mathcal{G}\Rightarrow \int_\mathcal{G}gdP=m(\mathcal{G})$) since integrating the conditional probability over $\mathcal{G}$ should yield 1 (by the Kolmogorov axioms). Also, the author is Robert Gray, not Gary. – sai Jun 6 '12 at 19:17
@sai Oops... :) – Vokram Jun 7 '12 at 8:40
@NateEldredge Sorry I still have some confusions... Is $g$ is a real-valued measurable function used to define $m(\cdot|\mathcal{G})$? But then why is it integrating g with another probability measure P? – Vokram Jun 7 '12 at 8:48

The starting point for abstract measure theoretic conditional probability is conditional expectation. Essentially, one uses the identity $P(A)=\mathbb{E}(1_A)$.

Now let $(\Omega,\mathcal{B},P)$ be a probability space, $f$ a random variable and $\mathcal{G}$ a sub-$\sigma$-algebra of $\mathcal{B}$. The conditional expectation of $f$ with respect to $\mathcal{G}$ is a $\mathcal{G}$-measurable function $\mathbb{E}_\mathcal{B}$ such that for all $G\in\mathcal{G}$ $$\int_G \mathbb{E}_\mathcal{B}~dP=\int_G f~dP.$$ The notion is not very intuitive, but the idea is the following: Since $\mathbb{E}_\mathcal{B}$ is $\mathcal{G}$-measurable, it uses only the information in $\mathcal{G}$. The integral condition says that $\mathbb{E}_\mathcal{B}$ "averages $f$ out" over sets in $\mathcal{G}$.

Now if we want to calculate the conditional probability of the event $H\in\mathcal{B}$ with respect to the sub-$\sigma$-algebra $\mathcal{G}$, we simply take the conditional expectation of the indiacator function $1_H$. Then, a conditional probability of $H$ with respect to $\mathcal{G}$ is a $\mathcal{G}$-measurable function $\mathbb{P}^H_\mathcal{G}$ such that for all $G\in\mathcal{G}$ $$\int_G \mathbb{P}^H_\mathcal{G}~dP=\int_G 1_H~dP.$$ Since $\int_G 1_H~dP=P(H\cap G)$, this can be rewritten as $$\int_G \mathbb{P}^H_\mathcal{G}~dP=P(H\cap G).$$

This is fairly standard material, so I assume the author made simply some typos. The $h$ is superflous and the $m$ should be $P$.

-
Thank you Michael! Please allow me some time to digest your answer. – Vokram Jun 7 '12 at 10:00
I think this answer is along the lines of what I wrote. – Michael Chernick Jun 7 '12 at 11:33
@MichaelChernick Great write up, this really helped me, as well. So, could we give a definition of "conditional probability of an event $H$ given $\mathcal{G}$", denoted $P(H | \mathcal{G})$, as the a.s. unique random variable such that $P(H | \mathcal{G})$ is $\mathcal{G}$-measurable and $\int_G P(H | \mathcal{G}) dP = P(H \cap G)$ for all $G \in \mathcal{G}$? – bcf Jul 6 '15 at 13:29

I think gh means g(h) that is g evaluated on the event h. Here h is taking the role of H. I think this is just a fancy way of say P(G⋂H) = P(H|G)P(G) for all G in script G. It is an integral because you are integrating over all values that G takes on i.e. ∫P(H|G=x)dP(x) = P(G⋂H).

-
Sorry Michael I am still a bit confuesed... Am I correct in thinking that $g$ is a real-valued measurable function and $m(\cdot|\mathcal{G})$ is a probability measure defined by $m(\cdot|\mathcal{G})=g$? And how is $m(\cdot|\mathcal{G})$ related to $m(\cdot)$? Does the auther different probability measures? – Vokram Jun 7 '12 at 8:38
Like you I am guessing about the notation. But it does seem that my idea fits. m is the notation for the probability measure and yes I think gh stands for P(H|G=x). – Michael Chernick Jun 7 '12 at 11:32