6
$\begingroup$

Given that X and Y are random variables show that:

$$E[E[X \mid Y]] = E[X]$$

I was thinking that I could use the definition of expected value (the summation one) to solve this, but when I tried I hit a wall. Thanks.

$\endgroup$
9
$\begingroup$

This is the Law of Total Expectation. The proof is as follows:

$$ \begin{align} E[E[X|Y]] &= E \left[ \sum_{x} x \cdot P(X = x | Y) \right] \\ &= \sum_y \left[ \sum_{x} x \cdot P(X = x | Y = y) \right] P(Y = y)\\ &= \sum_x x \sum_y P(X = x | Y = y) \cdot P(Y = y) \\ &= \sum_x x \sum_y P(X = x \, \text{and} \, Y = y) \\ &= \sum_x x \cdot P(X = x)\\ &= E[X] \end{align} $$

The non-discrete is a bit tougher I think. The Wikipedia link should help you out here.

$\endgroup$
7
$\begingroup$

The continuous case is similar. Let us specify the Law of Total Expectation (also called Tower Property) more precisely:

$$E_Y(E_X[X|Y])=E_X[X]$$

where $E_Y$ is the expectation w.r.t. $Y$ and $E_X$ w.r.t. $X$. Let $S_X=supp(X)$ and $S_Y=supp(Y)$ $$E_Y(E_X[X|Y])=\int_{S_Y} E_X[X|Y]\; f(y)\; dy$$ $$E_Y(E_X[X|Y])=\int_{S_Y}\int_{S_X}x\; \frac{f(x,y)}{f(y)}dx\; f(y)\; dy$$ $f(y)$ is constant w.r.t. $X$. Thus, $$E_Y(E_X[X|Y])=\int_{S_Y}\int_{S_X}x\; f(x,y)\;dx\; \frac{f(y)}{f(y)}\; dy$$ $$E_Y(E_X[X|Y])=\int_{S_Y}\int_{S_X}x\; f(x,y)\;dx\; dy$$ By Fubini's theorem $$E_Y(E_X[X|Y])=\int_{S_X}\int_{S_Y}x\; f(x,y)\;dy\; dx$$ Now $X$ is constant w.r.t. $Y$. $$E_Y(E_X[X|Y])=\int_{S_X}x\; \int_{S_Y} f(x,y)\;dy\; dx$$

The inner integral is the marginalisation from $f(x,y)$ to $f(x)$. Thus, $$E_Y(E_X[X|Y])=\int_{S_X}x\; f(x)\; dx$$ This is just the definition of the expectation. Thus, $$E_Y(E_X[X|Y])=E_X[X]$$

I hope this helps. This proof also exists in a measure theoretic context which is more general. The Wikipedia article is quite good.

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.