Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

$X$, $A$, and $Z$ are scalar independent random variables. $Y = AX + Z$. $$ A = \begin{cases}1 & \text{with probability } p \\ 0 & \text{with probability } 1-p\end{cases} $$ $X$ has mean $\mu$ and variance $\sigma_x^2$. $Z$ has mean $0$ and variance $\sigma_z^2$.

Find the optimal estimator of $X$ given that $Y$ is observed.

I know the optimal estimator to be $E\left[X\mid Y=y\right]$ but I don't know how to determine it.

I'm especially confused because if $A=0$ then $Y$ does not provide any information about $X$.

As always, thank you for your help.

share|cite|improve this question
Are you assuming $\mu$ is known, or must it be estimated based on the data? And do we know what $\sigma_X$ and $\sigma_Z$ are? I won't be surprised if a complete answer requires not only the expected values and variances of $X$ and $Z$ but further information about their distributions beyond that. – Michael Hardy Jan 31 '13 at 1:34
I just fixed a mistake in my answer: In the "linear regression" display, I had just $\rho$ where I should have had $\rho\sigma_X$. – Michael Hardy Feb 1 '13 at 17:29

$\newcommand{\E}{\mathbb E}$ $\newcommand{\var}{\operatorname{var}}$ $\newcommand{\cov}{\operatorname{cov}}$ \begin{align} \E(X\mid Y=y) & = \E( \E(X\mid Y=y, A)) \\[8pt] & = \E( X\mid Y=y, A=0)\Pr(A=0)+\E(X\mid Y=y,A=1)\Pr(A=1) \\[8pt] & = \E(X)(1-p)+ \E(X\mid X+Z=y)p.\tag{1} \end{align}

So we want the conditional expectation of $X$ given $X+Z$. We're not given their joint distribution, but we have at least some information about it. The correlation between $X$ and $X+Z$ is $$ \rho = \frac{\cov(X,X+Z)}{\sigma_X\sqrt{\sigma_X^2+\sigma_Z^2}} = \frac{\sigma_X^2}{\sigma_X\sqrt{\sigma_X^2+\sigma_Z^2}} = \frac{\sigma_X}{\sqrt{\sigma_X^2+\sigma_Z^2}}. $$ We can do a linear regression to find a predicted value $\widehat X$ of $X$ based on $X+Z$: $$ \widehat{X} = \mu_X +\rho\sigma_X\left( \frac{(X+Z)-(\mu_{X+Z})}{\sigma_{X+Z}} \right). $$ This would be the conditional expected value of $X$ given $X+Z$ if we knew that $X$ and $Z$ were normally distributed (given that they're independent; without that, we'd wonder whether joint normality is given). I think without further information about the distributions, we probably can't find this conditional expected value.

But what about "optimality" of estimation? I think one could show this has the smallest mean squared error of any estimator that is an affine function of the observed value of $X+Z$.

Then after this, we'd want the weighted average of this estimator and $\E(X)$, as mentioned in $(1)$ above.

share|cite|improve this answer
I just fixed a mistake: In the "linear regression" display above, I had just $\rho$ where I should have had $\rho\sigma_X$. – Michael Hardy Feb 1 '13 at 17:28

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.