Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

After some calculations I ended up with (there each $V_k : \mathbb{R} \to \mathbb{R}$)

$$ \sum_d \widetilde{P}_D(d) \left( \sum_{f' \in \mathcal{F}} P_{F|\;D}(f'|\;d) V_j(f',d) \right) \left( \sum_{f \in \mathcal{F}} P_{F|\;D}(f|\;d) V_i(f,d) \right) $$


$$ E_{P_{F|\;D}}[V_i|\;d] = \sum_{f \in \mathcal{F} } P_{F|\;D}(f|\;d) V_i(f,d) $$

I get

$$ \sum_d \widetilde{P}_D(d) E_{P_{F|\;D}}[V_j|\;d] E_{P_{F|\;D}}[V_i|\;d] $$

Which I believe equals

$$ E_{\widetilde{P}_d P_{F|\;D}}[V_i V_j] $$

and just rewriting the first expression it is equal to:

$$ \sum_d \widetilde{P}_D(d) \sum_{f \in \mathcal{F}} \sum_{f' \in \mathcal{F}} P_{F|\;D}(f|\;d) P_{F|\;D}(f'|\;d) V_j(f',d) V_i(f,d) $$

Which seems strange to me, that the expectation value of $V_i V_j$ would take into consideration the value when they are fed the same $d$ but different $f$.

I thought it would be written as

$$ \sum_d \widetilde{P}_D(d) \sum_{f \in \mathcal{F}} P_{F|\;D}(f|\;d) V_j(f,d) V_i(f,d) $$

Is it correct that the expression at the top equals $E_{\widetilde{P}_d P_{F|\;D}}[V_i V_j]$?

share|cite|improve this question
up vote 2 down vote accepted

The trouble lies in the step where you write Which I believe equals, and comes from the fact that for some random variables $X$, $Y$ and $Z$, in general, $$ \mathrm E(XY)\ne \mathrm E(\mathrm E(X\mid Z)\mathrm E(Y\mid Z)). $$ For a counterexample, assume that $X$ and $Y$ are i.i.d. and square integrable random variables and that $Z=\frac12(X+Y)$.

The computations below hold in this general setting but one can assume for simplicity that the random variables $X$ and $Y$ are two independent centered $\pm1$ Bernoulli random variables or, equivalently, that $(X,Y)$ is uniform on the set $\{-1,1\}^2$. Then $\mathrm E(X)=\mathrm E(Y)=\mathrm E(Z)=0$ and $\mathrm E(Z^2)=\frac12$.

Then, on the one hand, $\mathrm E(X\mid Z)=\mathrm E(Y\mid Z)=\frac12\mathrm E(X+Y\mid Z)=Z$ by symmetry, hence $$ \mathrm E(\mathrm E(X\mid Z)\mathrm E(Y\mid Z))=\mathrm E(Z^2). $$ On the other hand, $\mathrm E(XY)=\mathrm E(X)\mathrm E(Y)$ by independence and $\mathrm E(X)=\mathrm E(Y)=\mathrm E(Z)$ by symmetry again, hence $\mathrm E(XY)=\mathrm E(Z)^2$. One sees that for every nondegenerate distribution of $X$ and $Y$, $$ \mathrm E(\mathrm E(X\mid Z)\mathrm E(Y\mid Z))-\mathrm E(XY)=\text{Var}(Z)\ne0. $$ Edit For a case where $\mathrm E(\mathrm E(X\mid Z)\mathrm E(Y\mid Z))-\mathrm E(XY)\lt0$, consider $Z=\frac12(X-Y)$.

share|cite|improve this answer
Many thanks for this. – j-a Jan 24 '12 at 21:10

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.