Will the conditional expectation always have this “property”?(understanding/explanation of conditional expectation)

Question

Lets say you have a probability space $(\Omega, \mathcal{A},P)$, and a random variable $X: \Omega \rightarrow \mathbb{R}$ on this space. Assume that we have a sub-sigma algebra $\mathcal{G}\subset \mathcal{A}$. We can then show that $\mu_X(G)=\int_GXdP$, is a measure on $(\Omega, \mathcal{G})$, it is also easy to see that this measure is absolutely continuous with respect to P. The Radon-Nikodym theorem tells us that we have a unique $P$-a.e $\mathcal{G}$-measurable function $\mathcal{E}(X|\mathcal{G})$ on $\Omega$, s.t. $\mu_X(G)=\int_G\mathcal{E}(X|\mathcal{G})dP, G \in \mathcal{G}$.

My problem is that I have a hard time describing $\mathcal{E}(X|\mathcal{G})$ in general. If I make a specific example like this:

$\Omega=\{1,2,3,4\}$

$\mathcal{A}=\{\emptyset,\{1\},\{2,3\},\{4\},\{1,4\},\{1,2,3\},\{2,3,4\},\Omega\}$ $ P(\{1\})=0.5,P(\{2,3\})=0.25, P(\{4\})=0.25$

$ X(1)=1, X(2)=3,X(3)=3,X(4)=4$

$\mathcal{G}=\{\emptyset, \{1,4\},\{2,3\},\Omega\}$

Then a calculation, and using the uniqueness of the radon nikodym derivative, gives us that:

$\mathcal{E}(X|\mathcal{G})(\omega)=2\mathcal{X}_{\{1,4\}}(\omega)+3\mathcal{X}_{\{2,3\}}(\omega)$.

Now comes my question: From elementary courses in probability and statistics, we can show that $E(X|\{1,4\})=2$ and $E(X|\{2,3\})=3$.

So in this case, we see that the conditional expectation can be described this way: If you can only differentiate between the sets in $\mathcal{G}$ then the value of the conditional expectation for a given omega, is the value of the conditional expectation of the set in $\mathcal{G}$ containing $\Omega$ which is "smallest", or where we have eliminated most of the possibilities that did not happen, and that the sigma algebra $\mathcal{G}$ allows us to remove.

But this was an easy example. And in general you may not have smallest sets like this? But is there an intuitive or good explanation of the value of the conditional expectation when we work with larger sets like countable or uncountable? Is there an equivalent way of saying in these cases for associating the value of the conditional expectation with the conditional expectation of a set as given in elementary statistics? Or is there some theorem or explanation that generalises what I did above for "small" sets. And gives a good intuitive justification for conditional expectation here?

If you have other intuitive "explanations" of the conditional expectation, I would like to hear them as well.

Answer 1

If the $\sigma$-algebra $\mathcal G$ is generated by a finite or countably infinite partition $\{G_1,G_2,\ldots\}$ of $\Omega$ into $\mathcal A$-measurable sets, then the "naive" recipe $$ \Bbb E[X|\mathcal G] =\sum_n 1_{G_n}\Bbb E[X|G_n] $$ is valid. In general, we work with the conditional expectation by using its various properties, not by resorting to such a formula as displayed above. The beauty of Kolmogorov's construction of conditional expectation by means of the Radon-Nikodym theorem is that is works in complete generality.

If $\mathcal G$ is separable in the sense that $\mathcal G=\sigma\{A_1,A_2,\ldots\}$ for some sequence of $\mathcal A$-measurable sets then $\mathcal G_n:=\sigma\{A_1,\ldots, A_n\}$ is generated by a finite partition (disjointify the $A_k$s) and so $\Bbb E[X|\mathcal G_n]$ is given explicitly as in the display above. Moreover, $\Bbb E[X|\mathcal G_n]$ converges to $\Bbb E[X|\mathcal G]$ both almost surely and in $L^1$ as $n\to\infty$, by the Martingale Convergence Theorem. (This is the approach taken by John Walsh in his text Knowing the Odds.) You may find this construction more intuitive.

Even if $\mathcal G$ isn't separable, there is a separable $\sigma$-algebra $\mathcal G'\subset\mathcal G$ such that $\Bbb E[X|\mathcal G]$ is $\mathcal G'$-measurable, and then $\Bbb E[X|\mathcal G]=\Bbb E[X|\mathcal G']$ by the tower property of conditional expectations. The discussion of the preceding paragraph now applies. [For example, take $\mathcal G'$ to be generated by the countable collection $\{\omega\in\Omega: \Bbb E[X|\mathcal G](\omega)\le q\}$, $q\in\Bbb Q$, for some version of $\Bbb E[X|\mathcal G]$.]