# A few questions on conditional probability

I have a few questions relating to conditional expectation. It is covered very briefly (half a side of A4-size paper) in my lecture notes, but I feel like I should know about it a bit more in depth.

Essentially, the things I know are:

1. The definition. Given (i) a probability space $(\Omega, \mathcal{F}, \mathbb{P})$, (ii) a sub-$\sigma$-algebra $\mathcal{G} \subseteq \mathcal{F}$ which is generated by a countable set $(G_i\, :\, i \in I)$ with $\bigcup_i G_i = \Omega$, and (iii) a square-integrable random variable $X : \Omega \to \mathbb{R}$, we define the conditional expectation of $X$ given $\mathcal{G}$ by $\boxed{\mathbb{E}(X|\mathcal{G}) = \displaystyle \sum_{i \in I} \mathbb{E}(X|G_i)1_{G_i}}$, where $\mathbb{E}(X|G_i) = \dfrac{\mathbb{E}(X 1_{G_i})}{\mathbb{P}(G_i)}$.

2. The fact that $\mathbb{E}(X|\mathcal{G})$ is the orthogonal projection of $X$ onto the (closed, complete) subspace $L^2(\Omega, \mathcal{G}, \mathbb{P})$ of $L^2(\Omega, \mathcal{F}, \mathbb{P})$.

This is all fine. But a problem has arisen where I need to calculate this explicitly, and it's thrown me a bit. This has made me wonder a few things:

1. If $Y \in L^2$ is a random variable and $\mathcal{G}=\sigma(Y)$, we have an uncountable generating set for $\mathcal{G}$, namely $\{ Y^{-1}(B)\, :\, B \in \mathcal{B}(\mathbb{R}) \}$. But if $\mathcal{A} \subseteq \mathcal{B}(\mathbb{R})$ generates $\mathcal{B}(\mathbb{R})$, can we therefore take $\sigma(G) = \{ Y^{-1}(A)\, :\, A \in \mathcal{A} \}$? I'd presume we can since $Y^{-1}$ preserves set operations. If so, this becomes compatible with the above definition because we can choose a countable generating set, such as $\{ [a,b)\, :\, a,b \in \mathbb{Q},\ a < b \}$. Is this correct?

2. Do we define $\mathbb{E}(X|Y) = \mathbb{E}(X|\sigma(Y))$ for random variables $Y$? If not, how is this defined?

3. In Q25J on page 14 here (PDF) we have $(G,X) \sim N\left( (\mu, \nu), \begin{pmatrix} u & v \\ v & w \end{pmatrix} \right)$, $\mathcal{F} = \sigma(G,X)$ and $\mathcal{G} = \sigma(X)$. It is slightly unclear when it says to "find $Y$ explicitly in this case" $-$ I presume that it means I should find $\mathbb{E}( U\, |\, \mathcal{G})$ for a general $\mathcal{F}$-measurable $U \in L^2$, since this is (an) orthogonal projection. But I can't work out what I can say about the distribution of $Y$ in this case.

4. If $\mathcal{F} = \sigma(X)$ or $\sigma(X,Y)$ or so on, what can I say about the distribution of an $\mathcal{F}$-measurable function?

I hope I'm not trying to ask too much. Any input at all would be appreciated, so feel free to reply to as small or large a portion of my post as you like.

-
You are apparently seeing a very abstract version of this without having seen the elementary theory. I learned my first probabilty out of Chung, and came away with the impression that is was near impossible to construct conditional distributions. Every one else will do the particular problem you mention using that the conditional expectation is the mean of the conditional distribution, which is an easy thing to calculate for joint normals. Open Ross' intro to prob and read his chapters on it, whic will take 10 minutes. – mike Apr 30 '12 at 12:11
@mike: I've done elementary probability theory before, so I know what $\mathbb{P}(A|B)$ and $\mathbb{E}(X|Y)$ and so on mean from that perspective. The measure-theoretic twist is doing its bit to make my head explode! – Clive Newstead Apr 30 '12 at 18:35
for that exact problem, the normal, you can do the "projection " version by observing that if $X ,Y$ joint mormal you can write $Y = a X + Z$ where $Z$ is independent of $X$. This easy because for joint normal independent = uncorrelated. Then it is easy to see what $\mathbb E(Y \vert X)$ is . – mike Apr 30 '12 at 19:27

3. for that exact problem, the joint normal, you can do the "projection " version by observing that if $X ,Y$ joint normal you can write $Y = a X + Z$ where $Z$ is independent of $X$. This easy because for joint normal independent = uncorrelated. Then it is easy to see what $\mathbb E(Y \vert X)$ is . The projection formulation shows that if $X, Z$ independent, $\mathbb E(Z \vert X) = \mathbb E(Z)$. You can prove this as follows: for any function of X, by independence $\mathbb E(Zg(x)) = \mathbb E(Z) \mathbb E(g(x))$ so $\mathbb E((Z - \mathbb E(Z))g(x)) = 0$, so $Z - \mathbb E(Z)$ is orthogonal to any function of X, and that make $\mathbb E(Z)$ the projection of $Z$ on the space of $\sigma(X)$ measurable functions.