I am trying to figure out conditional expectation for the following case: Suppose $\theta$ has normal distribution with mean $0$ and variance $1$ i.e., standard normal. Let $x_i=\theta+\epsilon_i$ where $\epsilon_i$ has normal distribution with mean $0$ and variance $\frac{1-v}{v}$ for some $v\in(0,1)$. Given this it is claimed $\mathbf{E}[\theta\mid x_i]=vx_i$. To my mind, since $\theta=x_i-\epsilon_i$, thus $\mathbf E[\theta\mid x_i]=x_i$. To obtain answer I tried proper way which is using pdf of normal and expectation but I couldn't figure out how to solve integrals. Thanks for any help!

P.S. After a comment, $\theta$ and $\epsilon_i$ independent.

  • 1
    $\begingroup$ $\theta$ and $\epsilon_i$ are independent? $\endgroup$ – leonbloy May 13 '14 at 12:22
  • $\begingroup$ In the sentence "To my mind..." you make the common mistake of assuming $\mathbf E[\epsilon_i \mid x_i]=0$. That's false. $\endgroup$ – leonbloy May 13 '14 at 12:26
  • 1
    $\begingroup$ Why is it false? $\endgroup$ – user64066 May 13 '14 at 12:50

Sketch: Let $X=Y+Z$ with $Y,Z$ normal and independent: $Y \sim N(0,1) $ and $Z \sim N(0,a) $ We want $E[Y|X]$.

A common mistake (that you did) is to write $E[Y|X]=E[X|X]-E[Z|X]=X$ . The problem is that $X$ and $Z$ are not independent, hence $E[Z|X] \ne 0$ [*]

One way is to apply the inversion formula to the densities. Informally:

$$P(Y|X) = \frac{P(X|Y)P(Y)}{P(X)}=\frac{\psi(X;Y,a) \,\psi(Y;0,1)}{\psi(X;0,a+1)}$$ where $\psi(t;u,s)$ is the normal density function with mean $u$ and variance $s$ evaluated at $t$. Then, assuming $X$ is fixed, we can ignore the denominator (it's a normalization factor) and the numerator is the product of normals; working out the exponent (remember that $Y$ is our variable here) when can complete the square...

$$-\frac{(X-Y)^2}{2a}-\frac{Y^2}{2}=-\frac{1}{2a}\left((1+a)Y^2-2XY+X^2\right)=\\ =-\frac{1+a}{2a}\left( Y- \frac{X}{1+a} \right)^2 + c$$ with some $c$ does not depend on $Y$ (no need to compute it).

Hence, we get that $Y$ conditioned on $X$ is normal with mean $\frac{X}{1+a}$ (you also can see the variance), so $$E[Y|X]=\frac{X}{1+a}$$

[*] Why this mistake is common? Perhaps because of a wrong use of cause-effect concepts. Given $C=A+B$ one regards $C$ as an "ouput" and $A,B$ as "inputs" (ok), then slip on to consider of $A,B$ as "causes" and $C$ as the "effect", then one considers that "the output (effect) depends on the input (cause)", but on the contrary, "the input does not depend on the output" (false; for one thing, independence is a symmetric relation). Hence to condition on the output does not changes the probabilities of the inputs, and $E(A|C)=E(A)$. Of course, this is wrong.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.