I have a particular confusion on how to deal with an expectation of three variables where two are independent and the third is independent to one, but perhaps not (clearly defined) to be independent of the second.

Signal $X$ and noise $Z$ are independent random variables, where $$X=\begin{cases}+1\quad\text{with probability}\,\frac12\\-1\quad\text{with probability}\,\frac12\end{cases}$$ and $Z\sim U[-1,1]$. We observe $Y=gX+Z$, where $g$ itself is a random variable independent of both $X$ and $Y$. $$g=\begin{cases}+1\quad\text{with probability}\,p\\0\quad\text{with probability}\,1-p\end{cases}$$

(a) Find the best linear MMSE estimate of $X$ after observing $Y$, your answer should be in terms of $p$.

(b) Find the best (nonlinear) MMSE estimate of $X$ after observing $Y$, your answer should be in terms of $p$.

The formula for MMSE is $$\hat x= \frac{\operatorname{cov}(X, Y)}{\operatorname{var}(Y)}\cdot(Y-\Bbb E[Y])+\Bbb E[X].$$

What I've found so far is that $$\Bbb E[X] = 0,\quad\Bbb E[Y] = 0,\quad\Bbb E[XY] = p,$$ but then the part I got stuck on is $\Bbb E[Y^2]$ as $$\Bbb E[Y^2] = \Bbb E[(gX+Z)^2] = \Bbb E[g^2X^2 + 2gXZ + Z^2].$$

My Problem: Even if my math is incorrect (please let me know if it is), how do we deal with $2gXZ$ if $g$ is not independent to $Z$? Is there a property I am forgetting?

My Attempt: Otherwise, by brute force, I would simply follow through with calculating the three summations, correct? By definition, this should be $$2\sum_x\sum_z\sum_g gzxP_gP_zP_x.$$

How would one go about solving this? Wouldn't this result end up being "independent" anyways because each summation and random variable are independent of the indices? Also, I feel like we can only use this method because we were given that the distribution of $g$ is not a function of $X$ or $Z$ or $Y$?

Other thoughts

Note: That if $g$ is in fact independent to $Z$, does the logic here follow from the same logic as this post? Thus, we would say that since $Y$ is a function of $g$, $X$ and $Z$ and that $g$ is independent to $Y$, therefore $g$ is also independent to $X$ and $Z$? So it was redundant to say that $g$ was independent to $X$ in the first place?

Note: However, I feel that this problem has a slightly different flavor and I can't exactly pinpoint the intuition or reasoning to fully understand this problem.


Note that $X\perp g,Z$ so $\Bbb E[2gXZ]=2\Bbb E[X]\Bbb E[gZ]=0$ since $\Bbb E[X]=0$.

| cite | improve this answer | |
  • $\begingroup$ I guess this (sort of) answers my question? But Lets say that the term did not happen to go to zero. Is there a way to deal with this sort of problem? $\endgroup$ – CLDuser2.- Nov 23 '19 at 10:49
  • 1
    $\begingroup$ It is possible. In the continuous case, $\Bbb E[AB]=\iint abf_{A,B}(a,b)\,da\,db$. An analogous definition exists for the discrete case. $\endgroup$ – TheSimpliFire Nov 23 '19 at 10:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.