Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

After some calculations I ended up with (there each $V_k : \mathbb{R} \to \mathbb{R}$)

$$ \sum_d \widetilde{P}_D(d) \left( \sum_{f' \in \mathcal{F}} P_{F|\;D}(f'|\;d) V_j(f',d) \right) \left( \sum_{f \in \mathcal{F}} P_{F|\;D}(f|\;d) V_i(f,d) \right) $$

Since

$$ E_{P_{F|\;D}}[V_i|\;d] = \sum_{f \in \mathcal{F} } P_{F|\;D}(f|\;d) V_i(f,d) $$

I get

$$ \sum_d \widetilde{P}_D(d) E_{P_{F|\;D}}[V_j|\;d] E_{P_{F|\;D}}[V_i|\;d] $$

Which I believe equals

$$ E_{\widetilde{P}_d P_{F|\;D}}[V_i V_j] $$

and just rewriting the first expression it is equal to:

$$ \sum_d \widetilde{P}_D(d) \sum_{f \in \mathcal{F}} \sum_{f' \in \mathcal{F}} P_{F|\;D}(f|\;d) P_{F|\;D}(f'|\;d) V_j(f',d) V_i(f,d) $$

Which seems strange to me, that the expectation value of $V_i V_j$ would take into consideration the value when they are fed the same $d$ but different $f$.

I thought it would be written as

$$ \sum_d \widetilde{P}_D(d) \sum_{f \in \mathcal{F}} P_{F|\;D}(f|\;d) V_j(f,d) V_i(f,d) $$

Is it correct that the expression at the top equals $E_{\widetilde{P}_d P_{F|\;D}}[V_i V_j]$?

share|improve this question
add comment

1 Answer

up vote 2 down vote accepted

The trouble lies in the step where you write Which I believe equals, and comes from the fact that for some random variables $X$, $Y$ and $Z$, in general, $$ \mathrm E(XY)\ne \mathrm E(\mathrm E(X\mid Z)\mathrm E(Y\mid Z)). $$ For a counterexample, assume that $X$ and $Y$ are i.i.d. and square integrable random variables and that $Z=\frac12(X+Y)$.

The computations below hold in this general setting but one can assume for simplicity that the random variables $X$ and $Y$ are two independent centered $\pm1$ Bernoulli random variables or, equivalently, that $(X,Y)$ is uniform on the set $\{-1,1\}^2$. Then $\mathrm E(X)=\mathrm E(Y)=\mathrm E(Z)=0$ and $\mathrm E(Z^2)=\frac12$.

Then, on the one hand, $\mathrm E(X\mid Z)=\mathrm E(Y\mid Z)=\frac12\mathrm E(X+Y\mid Z)=Z$ by symmetry, hence $$ \mathrm E(\mathrm E(X\mid Z)\mathrm E(Y\mid Z))=\mathrm E(Z^2). $$ On the other hand, $\mathrm E(XY)=\mathrm E(X)\mathrm E(Y)$ by independence and $\mathrm E(X)=\mathrm E(Y)=\mathrm E(Z)$ by symmetry again, hence $\mathrm E(XY)=\mathrm E(Z)^2$. One sees that for every nondegenerate distribution of $X$ and $Y$, $$ \mathrm E(\mathrm E(X\mid Z)\mathrm E(Y\mid Z))-\mathrm E(XY)=\text{Var}(Z)\ne0. $$ Edit For a case where $\mathrm E(\mathrm E(X\mid Z)\mathrm E(Y\mid Z))-\mathrm E(XY)\lt0$, consider $Z=\frac12(X-Y)$.

share|improve this answer
    
Many thanks for this. –  j-a Jan 24 '12 at 21:10
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.