I tried to solve this exercise but got stuck: Assume we have the random variables $X$ and $Y$ where $E(X) = 0$. How can we prove the following inequality $\operatorname{Var} (X-E(X|Y)) \leq\operatorname{Var}(X)$?

I tried to write out the rhs: $\operatorname{Var}(X-E(X|Y)) = \operatorname{Var}(X) + \operatorname{Var}(E(X|Y)) - 2\operatorname{Cov}(X,E(X|Y))$ Since $E(X) = 0$, we have $\operatorname{Var}(E(X|Y)) = E(E(X|Y)^2)$ and $\operatorname{Cov}(X,E(X|Y)) = E(X*E(X|Y))$.

Thus we have $\operatorname{Var}(X) + E(E(X|Y)^2) - 2E(XE(X|Y)) \leq \operatorname{Var}(X)$ giving $E(E(X|Y)^2) \leq 2E(XE(X|Y))$

But that did not get me anywhere as you can see, does anybody have any tips? OR do you think there might be a typo in the exercise?

  • 3
    $\begingroup$ Hint: Start rather from $$X=(X-E(X\mid Y))+E(X\mid Y)$$ hence $$\mathrm{var}(X)=\ldots$$ $\endgroup$ – Did Aug 17 '17 at 15:10
  • $\begingroup$ Tnx for the reply! However, it didnt get me anywhere. I tried expanding, but nothing cancelled, care to give another tip? =)) $Var((X-E(X|Y)) + E(X|Y)) = Var(X-E(X|Y)) + Var(E(X|Y)) -2Cov(X-E(X|Y),E(X|Y)) $ $\endgroup$ – LearningMath Aug 18 '17 at 11:20
  • 1
    $\begingroup$ Let us turn to the covariance then (with a factor $+2$ instead of $-2$), which involves three terms: $$E((X-E(X\mid Y))E(X\mid Y))\qquad E(X-E(X\mid Y))\qquad E(E(X\mid Y))$$ What can one say about each of these terms? $\endgroup$ – Did Aug 18 '17 at 13:17
  • $\begingroup$ Opps +2 and not -2 , as you said, however, i still do not see how this track gets me anywhere: We have $E(X-E(X|Y)) = 0$ and $E(E(X|Y)) = 0$ and finally $Cov(X-E(X|Y),E(X|Y)) = E((X-E(X|Y))E(X|Y)) = E(XE(X|Y))-E(E(X|Y)^2)$ But this will only yield my original $E(E(X|Y)^2)<2E(XE(X|Y))$ :( $\endgroup$ – LearningMath Aug 18 '17 at 14:04
  • 1
    $\begingroup$ What you are lacking is a characterization of $E(X\mid Y)$ (until now, everything you tried would work with another random variable instead, which is a bad sign, right?). So, what is specific about $Z=E(X\mid Y)$, which makes that $E(XW)=E(ZW)$ for every $W$ such that $____$? IOW, how do you define $Z$? $\endgroup$ – Did Aug 18 '17 at 14:41

@Did's hints are spot on and you don't need $E(X)=0$.

Let $Z=E(X|Y)$, consider $$ \text{Var}(X)=\text{Var}(X-Z+Z)=\text{Var}(X-Z)+\text{Var}(Z)+2\text{Cov}(X-Z,Z). $$ Note that \begin{gather} E(X-Z|Y)=E(X-E(X|Y)|Y)=E(X|Y)-E(X|Y)=0\\ \implies E(X-Z)=0\implies \text{Cov}(X-Z,Z)=E[(X-Z)Z]. \end{gather} In going from the first line to the second line, we have used the Law of Iterated Expectations: $$ E(X-Z)=E[E(X-Z|Y)]=E(0)=0. $$ For $E[(X-Z)Z]$, we can condition on $Y$ again: $$ E[(X-Z)Z|Y]=ZE(X|Y)-Z^2=Z^2-Z^2=0\implies E[(X-Z)Z]=0. $$ Thus, we have $$ \text{Var}(X)=\text{Var}(X-Z)+\text{Var}(Z)\geq\text{Var}(X-Z)=\text{Var}[X-E(X|Y)]. $$

  • $\begingroup$ I think the OP has problems understanding the implication: $E(X-Z|Y) =0 \Rightarrow E(X-Z)=0$. $\endgroup$ – Harto Saarinen Nov 17 '17 at 8:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.