Let $X : \Omega \to S \subset \mathbb{R}$ and $Y : \Omega \to T \subset \mathbb{R}$ be random variables on $(\Omega, \mathcal{F}, P)$. Form the conditional expectation $$ E(Y \mid X) := E(Y \mid \sigma(X)). $$ How can one compute the variance of this random variable, directly from the definitions? Let's say $E(Y) = 0$ to simplify things. Here's my attempt: \begin{align*} \operatorname{Var}(E(Y \mid X)) & = E\left(\left(E(Y\mid X)\right)^2\right) \\ & = \int_{\Omega}\Big(E(Y\mid X(\omega))\Big)^2\, P(d\omega) \\ & = \int_{\Omega}\Big( \int_\Omega Y(\omega^\prime)\, P\big(d\omega^\prime \mid X(\omega)\big)\Big)^2\, P(d\omega) \\ & = \int_{S}\Big( \int_\Omega Y(\omega^\prime)\, P\big(d\omega^\prime \mid x\big)\Big)^2\, \Lambda_X(dx) \\ & = \int_{S}\Big( \int_T y\, \Lambda_{Y\mid X}\big(dy \mid x\big)\Big)^2\, \Lambda_X(dx) \\ & = \int_{S}\Big( \int_T y\, \Lambda_{Y\mid X}\big(dy \mid x\big)\Big)^2\, g(x)\, dx \\ & = \int_{S}\Big( \int_T y \,h(y \mid x)\, dy\Big)^2\, g(x)\, dx. \end{align*}

This first line is the definition of variance when $E(Y) = 0$. The second equality is the definition of expectation. The third equality is the definition of conditional expectation, where $P(\cdot \mid \cdot)$ is the regular conditional probability given $X$. The fourth equality is the change of variables formula, where $\Lambda_X$ is the distribution of $X$. The fifth line is also change of variables, where $\Lambda_{Y \mid X}$ is the conditional distribution of $Y$ given $X = x$. Finally, the last two lines are from Radon-Nikodym, where $g$ is the pdf for $X$ and $h(\cdot \mid x)$ is the conditional pdf of $Y$ given $X = x$. The variance is now a Riemann integral which we may compute.

Is this correct?

  • $\begingroup$ Assuming all the densities you postulate do exist, the final line of the displayed equations is correct. However, some intermediate steps are not, for example, the identity $$E\left(\left(E(Y|X)\right)^2\right) = \int_{\Omega}\Big(E(Y|X(\omega))\Big)^2\, P(d\omega)$$ is absurd since $E(Y\mid x)$ does not exist for $x$ in $S$ (except perhaps to denote $E(Y)$, which is not what you have in mind). A correct identity is $$E\left(\left(E(Y|X)\right)^2\right) = \int_{\Omega}\Big(E(Y|X)(\omega)\Big)^2\, P(d\omega),$$ which, sadly, does not help much. $\endgroup$ – Did May 12 '16 at 15:49
  • $\begingroup$ @Did Thanks. What would be the proper method to obtain line 3 from line 2, then? $\endgroup$ – bcf May 12 '16 at 15:56
  • $\begingroup$ There is none, I am afraid, since next to this, one also unnecessarily summons $P(d\omega'\mid x)$. Instead, one could jump directly from the RHS of line 1 to line 5 since the deduction is direct once one assumes the existence of $\Lambda_{Y\mid X}$ and $\Lambda_X$. $\endgroup$ – Did May 12 '16 at 16:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.