I was reading a book on information theory and entropy by Robert Gray, when I saw the following definition of conditional probability:

Given a probability space $(\Omega,\mathcal{B}, P)$ and a sub-$\sigma$-field $\mathcal{G}$, for any event $H\in\mathcal{B}$ the conditional probability $m(H\text{ }|\text{ }\mathcal{G})$ is defined as any function , say $g$, which satisfies the two properties:

(1) $g$ is measurable with respect to $\mathcal{G}$

(2) $\displaystyle\int_{G}ghdP=m(G\bigcap{}H)$; all $G\in\mathcal{G}$

I am quite confused with this definition since it is very different from the definition through joint probability of events.

I understand what measurable function, sub-$\sigma$-field and probability space are, and I'm guessing that the author is trying to definie the measure $m$ through the measurable function $g$, but I don't quite understand what the second requirement is saying. Especially, what does that h in $\displaystyle\int_{G}ghdP$ refer to? it just jumped out of nowhere in the book, so I'm suspecting that it may have some conventional meaning?

I'd appreciate it a lot if someone can help. Thank you!!

  • 1
    $\begingroup$ I think the $h$ is a typo and should not be there at all. $\endgroup$ – Nate Eldredge Jun 6 '12 at 18:43
  • $\begingroup$ Umm .. perhaps you want $h=m(\mathcal{G})$ for normalization (if $h=1$, then $H=G=\mathcal{G}\Rightarrow \int_\mathcal{G}gdP=m(\mathcal{G})$) since integrating the conditional probability over $\mathcal{G}$ should yield 1 (by the Kolmogorov axioms). Also, the author is Robert Gray, not Gary. $\endgroup$ – sai Jun 6 '12 at 19:17
  • $\begingroup$ @sai Oops... :) $\endgroup$ – Vokram Jun 7 '12 at 8:40
  • $\begingroup$ @NateEldredge Sorry I still have some confusions... Is $g$ is a real-valued measurable function used to define $m(\cdot|\mathcal{G})$? But then why is it integrating g with another probability measure P? $\endgroup$ – Vokram Jun 7 '12 at 8:48

The starting point for abstract measure theoretic conditional probability is conditional expectation. Essentially, one uses the identity $P(A)=\mathbb{E}(1_A)$.

Now let $(\Omega,\mathcal{B},P)$ be a probability space, $f$ a random variable and $\mathcal{G}$ a sub-$\sigma$-algebra of $\mathcal{B}$. The conditional expectation of $f$ with respect to $\mathcal{G}$ is a $\mathcal{G}$-measurable function $\mathbb{E}_\mathcal{B}$ such that for all $G\in\mathcal{G}$ $$\int_G \mathbb{E}_\mathcal{B}~dP=\int_G f~dP.$$ The notion is not very intuitive, but the idea is the following: Since $\mathbb{E}_\mathcal{B}$ is $\mathcal{G}$-measurable, it uses only the information in $\mathcal{G}$. The integral condition says that $\mathbb{E}_\mathcal{B}$ "averages $f$ out" over sets in $\mathcal{G}$.

Now if we want to calculate the conditional probability of the event $H\in\mathcal{B}$ with respect to the sub-$\sigma$-algebra $\mathcal{G}$, we simply take the conditional expectation of the indiacator function $1_H$. Then, a conditional probability of $H$ with respect to $\mathcal{G}$ is a $\mathcal{G}$-measurable function $\mathbb{P}^H_\mathcal{G}$ such that for all $G\in\mathcal{G}$ $$\int_G \mathbb{P}^H_\mathcal{G}~dP=\int_G 1_H~dP.$$ Since $\int_G 1_H~dP=P(H\cap G)$, this can be rewritten as $$\int_G \mathbb{P}^H_\mathcal{G}~dP=P(H\cap G).$$

This is fairly standard material, so I assume the author made simply some typos. The $h$ is superflous and the $m$ should be $P$.

| cite | improve this answer | |
  • $\begingroup$ Thank you Michael! Please allow me some time to digest your answer. $\endgroup$ – Vokram Jun 7 '12 at 10:00
  • $\begingroup$ I think this answer is along the lines of what I wrote. $\endgroup$ – Michael R. Chernick Jun 7 '12 at 11:33
  • $\begingroup$ @MichaelChernick Great write up, this really helped me, as well. So, could we give a definition of "conditional probability of an event $H$ given $\mathcal{G}$", denoted $P(H | \mathcal{G})$, as the a.s. unique random variable such that $P(H | \mathcal{G})$ is $\mathcal{G}$-measurable and $\int_G P(H | \mathcal{G}) dP = P(H \cap G)$ for all $G \in \mathcal{G}$? $\endgroup$ – bcf Jul 6 '15 at 13:29

I think gh means g(h) that is g evaluated on the event h. Here h is taking the role of H. I think this is just a fancy way of say P(G⋂H) = P(H|G)P(G) for all G in script G. It is an integral because you are integrating over all values that G takes on i.e. ∫P(H|G=x)dP(x) = P(G⋂H).

| cite | improve this answer | |
  • $\begingroup$ Sorry Michael I am still a bit confuesed... Am I correct in thinking that $g$ is a real-valued measurable function and $m(\cdot|\mathcal{G})$ is a probability measure defined by $m(\cdot|\mathcal{G})=g$? And how is $m(\cdot|\mathcal{G})$ related to $m(\cdot)$? Does the auther different probability measures? $\endgroup$ – Vokram Jun 7 '12 at 8:38
  • $\begingroup$ Like you I am guessing about the notation. But it does seem that my idea fits. m is the notation for the probability measure and yes I think gh stands for P(H|G=x). $\endgroup$ – Michael R. Chernick Jun 7 '12 at 11:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.