16
$\begingroup$

I was reading a book on information theory and entropy by Robert Gray, when I saw the following definition of conditional probability:

Given a probability space $(\Omega,\mathcal{B}, P)$ and a sub-$\sigma$-field $\mathcal{G}$, for any event $H\in\mathcal{B}$ the conditional probability $m(H\text{ }|\text{ }\mathcal{G})$ is defined as any function , say $g$, which satisfies the two properties:

(1) $g$ is measurable with respect to $\mathcal{G}$

(2) $\displaystyle\int_{G}ghdP=m(G\bigcap{}H)$; all $G\in\mathcal{G}$

I am quite confused with this definition since it is very different from the definition through joint probability of events.

I understand what measurable function, sub-$\sigma$-field and probability space are, and I'm guessing that the author is trying to definie the measure $m$ through the measurable function $g$, but I don't quite understand what the second requirement is saying. Especially, what does that h in $\displaystyle\int_{G}ghdP$ refer to? it just jumped out of nowhere in the book, so I'm suspecting that it may have some conventional meaning?

I'd appreciate it a lot if someone can help. Thank you!!

$\endgroup$
4
  • 1
    $\begingroup$ I think the $h$ is a typo and should not be there at all. $\endgroup$ Jun 6, 2012 at 18:43
  • $\begingroup$ Umm .. perhaps you want $h=m(\mathcal{G})$ for normalization (if $h=1$, then $H=G=\mathcal{G}\Rightarrow \int_\mathcal{G}gdP=m(\mathcal{G})$) since integrating the conditional probability over $\mathcal{G}$ should yield 1 (by the Kolmogorov axioms). Also, the author is Robert Gray, not Gary. $\endgroup$
    – sai
    Jun 6, 2012 at 19:17
  • $\begingroup$ @sai Oops... :) $\endgroup$
    – Vokram
    Jun 7, 2012 at 8:40
  • $\begingroup$ @NateEldredge Sorry I still have some confusions... Is $g$ is a real-valued measurable function used to define $m(\cdot|\mathcal{G})$? But then why is it integrating g with another probability measure P? $\endgroup$
    – Vokram
    Jun 7, 2012 at 8:48

2 Answers 2

28
$\begingroup$

The starting point for abstract measure theoretic conditional probability is conditional expectation. Essentially, one uses the identity $P(A)=\mathbb{E}(1_A)$.

Now let $(\Omega,\mathcal{B},P)$ be a probability space, $f$ a random variable and $\mathcal{G}$ a sub-$\sigma$-algebra of $\mathcal{B}$. The conditional expectation of $f$ with respect to $\mathcal{G}$ is a $\mathcal{G}$-measurable function $\mathbb{E}_\mathcal{B}$ such that for all $G\in\mathcal{G}$ $$\int_G \mathbb{E}_\mathcal{B}~dP=\int_G f~dP.$$ The notion is not very intuitive, but the idea is the following: Since $\mathbb{E}_\mathcal{B}$ is $\mathcal{G}$-measurable, it uses only the information in $\mathcal{G}$. The integral condition says that $\mathbb{E}_\mathcal{B}$ "averages $f$ out" over sets in $\mathcal{G}$.

Now if we want to calculate the conditional probability of the event $H\in\mathcal{B}$ with respect to the sub-$\sigma$-algebra $\mathcal{G}$, we simply take the conditional expectation of the indicator function $1_H$. Then, a conditional probability of $H$ with respect to $\mathcal{G}$ is a $\mathcal{G}$-measurable function $\mathbb{P}^H_\mathcal{G}$ such that for all $G\in\mathcal{G}$ $$\int_G \mathbb{P}^H_\mathcal{G}~dP=\int_G 1_H~dP.$$ Since $\int_G 1_H~dP=P(H\cap G)$, this can be rewritten as $$\int_G \mathbb{P}^H_\mathcal{G}~dP=P(H\cap G).$$

This is fairly standard material, so I assume the author made simply some typos. The $h$ is superflous and the $m$ should be $P$.

$\endgroup$
3
  • $\begingroup$ Thank you Michael! Please allow me some time to digest your answer. $\endgroup$
    – Vokram
    Jun 7, 2012 at 10:00
  • $\begingroup$ I think this answer is along the lines of what I wrote. $\endgroup$ Jun 7, 2012 at 11:33
  • 1
    $\begingroup$ @MichaelChernick Great write up, this really helped me, as well. So, could we give a definition of "conditional probability of an event $H$ given $\mathcal{G}$", denoted $P(H | \mathcal{G})$, as the a.s. unique random variable such that $P(H | \mathcal{G})$ is $\mathcal{G}$-measurable and $\int_G P(H | \mathcal{G}) dP = P(H \cap G)$ for all $G \in \mathcal{G}$? $\endgroup$
    – bcf
    Jul 6, 2015 at 13:29
1
$\begingroup$

I think gh means g(h) that is g evaluated on the event h. Here h is taking the role of H. I think this is just a fancy way of say P(G⋂H) = P(H|G)P(G) for all G in script G. It is an integral because you are integrating over all values that G takes on i.e. ∫P(H|G=x)dP(x) = P(G⋂H).

$\endgroup$
2
  • $\begingroup$ Sorry Michael I am still a bit confuesed... Am I correct in thinking that $g$ is a real-valued measurable function and $m(\cdot|\mathcal{G})$ is a probability measure defined by $m(\cdot|\mathcal{G})=g$? And how is $m(\cdot|\mathcal{G})$ related to $m(\cdot)$? Does the auther different probability measures? $\endgroup$
    – Vokram
    Jun 7, 2012 at 8:38
  • $\begingroup$ Like you I am guessing about the notation. But it does seem that my idea fits. m is the notation for the probability measure and yes I think gh stands for P(H|G=x). $\endgroup$ Jun 7, 2012 at 11:32

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .