# Mean Square Error Optimal Estimator of X given Y

$X$, $A$, and $Z$ are scalar independent random variables. $Y = AX + Z$. $$A = \begin{cases}1 & \text{with probability } p \\ 0 & \text{with probability } 1-p\end{cases}$$ $X$ has mean $\mu$ and variance $\sigma_x^2$. $Z$ has mean $0$ and variance $\sigma_z^2$.

Find the optimal estimator of $X$ given that $Y$ is observed.

I know the optimal estimator to be $E\left[X\mid Y=y\right]$ but I don't know how to determine it.

I'm especially confused because if $A=0$ then $Y$ does not provide any information about $X$.

As always, thank you for your help.

-
Are you assuming $\mu$ is known, or must it be estimated based on the data? And do we know what $\sigma_X$ and $\sigma_Z$ are? I won't be surprised if a complete answer requires not only the expected values and variances of $X$ and $Z$ but further information about their distributions beyond that. – Michael Hardy Jan 31 '13 at 1:34
I just fixed a mistake in my answer: In the "linear regression" display, I had just $\rho$ where I should have had $\rho\sigma_X$. – Michael Hardy Feb 1 '13 at 17:29

$\newcommand{\E}{\mathbb E}$ $\newcommand{\var}{\operatorname{var}}$ $\newcommand{\cov}{\operatorname{cov}}$ \begin{align} \E(X\mid Y=y) & = \E( \E(X\mid Y=y, A)) \\[8pt] & = \E( X\mid Y=y, A=0)\Pr(A=0)+\E(X\mid Y=y,A=1)\Pr(A=1) \\[8pt] & = \E(X)(1-p)+ \E(X\mid X+Z=y)p.\tag{1} \end{align}
So we want the conditional expectation of $X$ given $X+Z$. We're not given their joint distribution, but we have at least some information about it. The correlation between $X$ and $X+Z$ is $$\rho = \frac{\cov(X,X+Z)}{\sigma_X\sqrt{\sigma_X^2+\sigma_Z^2}} = \frac{\sigma_X^2}{\sigma_X\sqrt{\sigma_X^2+\sigma_Z^2}} = \frac{\sigma_X}{\sqrt{\sigma_X^2+\sigma_Z^2}}.$$ We can do a linear regression to find a predicted value $\widehat X$ of $X$ based on $X+Z$: $$\widehat{X} = \mu_X +\rho\sigma_X\left( \frac{(X+Z)-(\mu_{X+Z})}{\sigma_{X+Z}} \right).$$ This would be the conditional expected value of $X$ given $X+Z$ if we knew that $X$ and $Z$ were normally distributed (given that they're independent; without that, we'd wonder whether joint normality is given). I think without further information about the distributions, we probably can't find this conditional expected value.
But what about "optimality" of estimation? I think one could show this has the smallest mean squared error of any estimator that is an affine function of the observed value of $X+Z$.
Then after this, we'd want the weighted average of this estimator and $\E(X)$, as mentioned in $(1)$ above.
I just fixed a mistake: In the "linear regression" display above, I had just $\rho$ where I should have had $\rho\sigma_X$. – Michael Hardy Feb 1 '13 at 17:28