# Intuitive explanation of the tower property of conditional expectation

I understand how to define conditional expectation and how to prove that it exists.

Further, I think I understand what conditional expectation means intuitively. I can also prove the tower property, that is if $X$ and $Y$ are random variables (or $Y$ a $\sigma$-field) then we have that

$$\mathbb E[X] = \mathbb{E}[\mathbb E [X | Y]].$$

My question is: What is the intuitive meaning of this? It seems quite puzzling to me.

(I could find similar questions but not this one.)

-

For simple discrete situations from which one obtains most basic intuitions, the meaning is clear.

I have a large bag of biased coins. Suppose that half of them favour heads, probability of head $0.7$. Two-fifths of them favour heads, probability of head $0.8$. And the rest favour heads, probability of head $0.9$.

Pick a coin at random, toss it, say once. To find the expected number of heads, calculate the expectations, given the various biasing possibilities. Then average the answers, taking into consideration the proportions of the various types of coin.

It is intuitively clear that this formal procedure "should" give about the same answer as the highly informal process of say repeating the experiment $1000$ times, and dividing by $1000$. For if we do that, in about $500$ cases we will get the first type of coin, and out of these $500$ we will get about $350$ heads, and so on. The informal arithmetic mirrors exactly the more formal process described in the preceding paragraph.

If it is more persuasive, we can imagine tossing the chosen coin $12$ times.

-

First, recall that in $E[X|Y]$ we are taking the expectation with respect to $X$, and so it can be written as $E[X|Y]=E_X[X|Y]=g(Y)$ . Because it's a funciton of $Y$, it's a random variable, and hence we can take its expectation (with respect to $Y$ now). So the double expectation should be read as $E_Y[E_X[X|Y]]$.

About the intuitive meaning, there are several approaches. I like to think of the expectation as a kind of predictor/guess (indeed, it's the predictor that minimizes the mean squared error).

Suppose for example that $X, Y$ are two (positively) correlated variables, say the weigth and height of persons from a given population. The expectation of the weight $E(X)$ would be my best guess of the weight of a unknown person: I'd bet for this value, if not given more data (my uninformed bet is constant). Instead, if I know the height, I'd bet for $E(X | Y)$ : that means that for different persons I'd bet a diferent value, and my informed bet would not be constant: sometimes I'd bet more that the "uninformed bet" $E(X)$ (for tall persons) , sometime less. The natural question arises, can I say something about my informed bet in average? Well, the tower property answers: In average, you'll bet the same.

-
The first paragraph contains undefined and potentially misleading expressions: I would not know how to define *the expectation with respect to $X$* nor the operator $E_X$ in the notation $E_X[X|Y]$ (likewise for similar terms based on $Y$). –  Did May 27 '11 at 20:24
$E_X()$ is just a notation to emphasize that we are integrating over the variable $X$. –  leonbloy May 27 '11 at 20:31
Precisely: the expression we are integrating over the variable $X$ means nothing (as opposed to the standard to integrate a random variable or to integrate over a set or to integrate with respect to a measure). –  Did May 27 '11 at 20:40
Since other people have already posted probabilistic explanations, I'll offer something different. If you're geometrically inclined, perhaps it's a little easier to explain it in terms of orthogonal projections. Recall that if $X$ is a $L^2$ random variable on the probability space $(\Omega, \mathcal{F}, \mathbb{P})$, and $\mathcal{G} \subseteq \mathcal{F}$ is a sub-$\sigma$-algebra, then $\mathbb{E}[X | \mathcal{G}]$ is just the orthogonal projection of $X$ onto the subspace $L^2(\Omega, \mathcal{G}, \mathbb{P}) \subseteq L^2(\Omega, \mathcal{F}, \mathbb{P})$. Suppose now that $\mathcal{L} \subseteq \mathcal{G}$ is another sub-$\sigma$-algebra, then, we have $$\mathbb{E}[\mathbb{E}[X | \mathcal{G}] | \mathcal{L}] = \mathbb{E}[X | \mathcal{L}]$$ simply because the composition of two orthogonal projections is another orthogonal projection. The usual expectation operator is just conditional expectation with respect to the trivial $\sigma$-algebra, so as a special case we obtain $$\mathbb{E}[\mathbb{E}[X | \mathcal{G}]] = \mathbb{E}[X]$$ as required.
The expected value of $X$ is still the expected value of $X$ when you take into account the possible values of $Y$.