Expected Value Proof - Law of Total Expectation. Given that X and Y are random variables show that:
$$E[E[X
\mid Y]] = E[X]$$
I was thinking that I could use the definition of expected value (the summation one) to solve this, but when I tried I hit a wall. Thanks.
 A: The continuous case is similar. Let us specify the Law of Total Expectation (also called Tower Property) more precisely:
$$E_Y(E_X[X|Y])=E_X[X]$$
where $E_Y$ is the expectation w.r.t. $Y$ and $E_X$ w.r.t. $X$. Let $S_X=supp(X)$ and $S_Y=supp(Y)$
$$E_Y(E_X[X|Y])=\int_{S_Y} E_X[X|Y]\; f(y)\; dy$$
$$E_Y(E_X[X|Y])=\int_{S_Y}\int_{S_X}x\; \frac{f(x,y)}{f(y)}dx\; f(y)\; dy$$
$f(y)$ is constant w.r.t. $X$. Thus,
$$E_Y(E_X[X|Y])=\int_{S_Y}\int_{S_X}x\; f(x,y)\;dx\; \frac{f(y)}{f(y)}\; dy$$
$$E_Y(E_X[X|Y])=\int_{S_Y}\int_{S_X}x\; f(x,y)\;dx\; dy$$
By Fubini's theorem
$$E_Y(E_X[X|Y])=\int_{S_X}\int_{S_Y}x\; f(x,y)\;dy\; dx$$
Now $X$ is constant w.r.t. $Y$.
$$E_Y(E_X[X|Y])=\int_{S_X}x\; \int_{S_Y} f(x,y)\;dy\; dx$$
The inner integral is the marginalisation from $f(x,y)$ to $f(x)$. Thus,
$$E_Y(E_X[X|Y])=\int_{S_X}x\; f(x)\; dx$$
This is just the definition of the expectation. Thus,
$$E_Y(E_X[X|Y])=E_X[X]$$
I hope this helps. This proof also exists in a measure theoretic context which is more general. The Wikipedia article is quite good.
A: This is the Law of Total Expectation. The proof is as follows:
$$ \begin{align}
E[E[X|Y]] &=  E \left[ \sum_{x} x \cdot P(X = x | Y) \right] \\
&= \sum_y \left[ \sum_{x} x \cdot P(X = x | Y = y) \right] P(Y = y)\\
&= \sum_x x \sum_y  P(X = x | Y = y) \cdot P(Y = y) \\
&= \sum_x x \sum_y P(X = x \, \text{and} \, Y = y) \\
&= \sum_x x \cdot P(X = x)\\
&= E[X]
\end{align}
$$
The non-discrete is a bit tougher I think. The Wikipedia link should help you out here.
