# Definition of Conditional Probability by Measure Theory

I was reading a book on information theory and entropy by Robert Gray, when I saw the following definition of conditional probability:

Given a probability space $(\Omega,\mathcal{B}, P)$ and a sub-$\sigma$-field $\mathcal{G}$, for any event $H\in\mathcal{B}$ the conditional probability $m(H\text{ }|\text{ }\mathcal{G})$ is defined as any function , say $g$, which satisfies the two properties:

(1) $g$ is measurable with respect to $\mathcal{G}$

(2) $\displaystyle\int_{G}ghdP=m(G\bigcap{}H)$; all $G\in\mathcal{G}$

I am quite confused with this definition since it is very different from the definition through joint probability of events.

I understand what measurable function, sub-$\sigma$-field and probability space are, and I'm guessing that the author is trying to definie the measure $m$ through the measurable function $g$, but I don't quite understand what the second requirement is saying. Especially, what does that h in $\displaystyle\int_{G}ghdP$ refer to? it just jumped out of nowhere in the book, so I'm suspecting that it may have some conventional meaning?

I'd appreciate it a lot if someone can help. Thank you!!

• I think the $h$ is a typo and should not be there at all. Jun 6, 2012 at 18:43
• Umm .. perhaps you want $h=m(\mathcal{G})$ for normalization (if $h=1$, then $H=G=\mathcal{G}\Rightarrow \int_\mathcal{G}gdP=m(\mathcal{G})$) since integrating the conditional probability over $\mathcal{G}$ should yield 1 (by the Kolmogorov axioms). Also, the author is Robert Gray, not Gary.
– sai
Jun 6, 2012 at 19:17
• @sai Oops... :) Jun 7, 2012 at 8:40
• @NateEldredge Sorry I still have some confusions... Is $g$ is a real-valued measurable function used to define $m(\cdot|\mathcal{G})$? But then why is it integrating g with another probability measure P? Jun 7, 2012 at 8:48

The starting point for abstract measure theoretic conditional probability is conditional expectation. Essentially, one uses the identity $$P(A)=\mathbb{E}(1_A)$$.

Now let $$(\Omega,\mathcal{B},P)$$ be a probability space, $$f$$ a random variable and $$\mathcal{G}$$ a sub-$$\sigma$$-algebra of $$\mathcal{B}$$. The conditional expectation of $$f$$ with respect to $$\mathcal{G}$$ is a $$\mathcal{G}$$-measurable function $$\mathbb{E}_\mathcal{B}$$ such that for all $$G\in\mathcal{G}$$ $$\int_G \mathbb{E}_\mathcal{B}~dP=\int_G f~dP.$$ The notion is not very intuitive, but the idea is the following: Since $$\mathbb{E}_\mathcal{B}$$ is $$\mathcal{G}$$-measurable, it uses only the information in $$\mathcal{G}$$. The integral condition says that $$\mathbb{E}_\mathcal{B}$$ "averages $$f$$ out" over sets in $$\mathcal{G}$$.

Now if we want to calculate the conditional probability of the event $$H\in\mathcal{B}$$ with respect to the sub-$$\sigma$$-algebra $$\mathcal{G}$$, we simply take the conditional expectation of the indicator function $$1_H$$. Then, a conditional probability of $$H$$ with respect to $$\mathcal{G}$$ is a $$\mathcal{G}$$-measurable function $$\mathbb{P}^H_\mathcal{G}$$ such that for all $$G\in\mathcal{G}$$ $$\int_G \mathbb{P}^H_\mathcal{G}~dP=\int_G 1_H~dP.$$ Since $$\int_G 1_H~dP=P(H\cap G)$$, this can be rewritten as $$\int_G \mathbb{P}^H_\mathcal{G}~dP=P(H\cap G).$$

This is fairly standard material, so I assume the author made simply some typos. The $$h$$ is superflous and the $$m$$ should be $$P$$.

• Thank you Michael! Please allow me some time to digest your answer. Jun 7, 2012 at 10:00
• I think this answer is along the lines of what I wrote. Jun 7, 2012 at 11:33
• @MichaelChernick Great write up, this really helped me, as well. So, could we give a definition of "conditional probability of an event $H$ given $\mathcal{G}$", denoted $P(H | \mathcal{G})$, as the a.s. unique random variable such that $P(H | \mathcal{G})$ is $\mathcal{G}$-measurable and $\int_G P(H | \mathcal{G}) dP = P(H \cap G)$ for all $G \in \mathcal{G}$?
– bcf
Jul 6, 2015 at 13:29

I think gh means g(h) that is g evaluated on the event h. Here h is taking the role of H. I think this is just a fancy way of say P(G⋂H) = P(H|G)P(G) for all G in script G. It is an integral because you are integrating over all values that G takes on i.e. ∫P(H|G=x)dP(x) = P(G⋂H).

• Sorry Michael I am still a bit confuesed... Am I correct in thinking that $g$ is a real-valued measurable function and $m(\cdot|\mathcal{G})$ is a probability measure defined by $m(\cdot|\mathcal{G})=g$? And how is $m(\cdot|\mathcal{G})$ related to $m(\cdot)$? Does the auther different probability measures? Jun 7, 2012 at 8:38
• Like you I am guessing about the notation. But it does seem that my idea fits. m is the notation for the probability measure and yes I think gh stands for P(H|G=x). Jun 7, 2012 at 11:32