Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I'm trying to derive a proof that, for discrete random variables $U$, $V$, $W$ with finite expectation, it holds that $$E[\ E[V\mid U,W]\mid W\ ] = E[V\mid W]. \qquad (1) $$ (I'm aware of the measure theory based proof, but I'm trying to find a proof that does without. The main difficulty I'm encountering is the additional conditioning on $W$; the proof of $E[ E[V\mid U] ] = E[V]$ is clear to me.)

Let $X(\Omega_X)$ be the range of RV $X$. The LHS of $(1)$ is itself a random variable, namely a function $f$ where \begin{align*} f(w_1) =& \sum_{(u,w) \in U(\Omega_U)\times V(\Omega_W)} E[\ V \mid U = u \wedge W = w]\ Pr[ U = u \wedge W = w \mid W = w_1 ]\\ =& \sum_{(u,w)} \sum_{v \in V(\Omega_V)} v \cdot Pr[ V = v \mid U = u \wedge W = w]\cdot Pr[ U = u \wedge W = w \mid W = w_1 ] \end{align*}

The RHS of $(1)$ is a function $g$ such that $$ g(w_1) = \sum_{v \in V(\Omega_V)} v \cdot Pr[\ V = v \mid W = w_1\ ]. $$ To proof $(1)$, we need to show that for all $w_1 \in W(\Omega_W)$ we have $$f(w_1) = g(w_1).$$ This is true if $$ Pr[ V = v \mid U = u \wedge W = w\ ] \cdot Pr[U = u \wedge W = w \mid W = w_1 ] = Pr[V = v \mid W = w_1 ], $$ but I couldn't progress from here. I have a feeling that my interpretation of the LHS is incorrect...

share|cite|improve this question
Are you considering the case when random every random variable $U,V,W$ takes only finitely many values? – Ilya Aug 31 '12 at 8:06
Not necessarily, but for now I would be grateful for a proof even for this case. – somebody Aug 31 '12 at 8:43
up vote 2 down vote accepted

This is true if $$ Pr[ V = v \mid U = u \wedge W = w\ ] \cdot Pr[U = u \wedge W = w \mid W = w_1 ] = Pr[V = v \mid W = w_1 ] $$

Not quite. In fact, this is true if, for every given $w_1$, $$ \sum_{u,w}\Pr[ V = v \mid U = u, W = w] \cdot \Pr[U = u, W = w \mid W = w_1 ] = \Pr[V = v \mid W = w_1 ]. $$ To prove this, fix $u$ and consider $a(w)=\Pr[U = u, W = w \mid W = w_1 ]$. If one refers to the definition of conditional expectation of events, one sees that $a(w)=0$ for every $w$ except $w=w_1$, and that $a(w_1)=\Pr[U = u\mid W = w_1 ]$. It follows that each sum over $w$ in the LHS of the identity to be proved has only one nonzero term, which is $$ \Pr[ V = v \mid U = u, W = w_1] \cdot \Pr[U = u\mid W = w_1 ]. $$ This is $$ \frac{\Pr[ V = v,U = u, W = w_1]}{\Pr[U = u, W = w_1]} \cdot \frac{\Pr[U = u,W = w_1 ]}{\Pr[W = w_1 ]}=\Pr[ V = v, U = u\mid W = w_1] . $$ Hence the whole LHS is $$ \sum_u\Pr[ V = v, U = u\mid W = w_1]=\Pr[ V = v\mid W = w_1], $$ as desired.

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.