Estimation of a random variable X given another random variable Y but X and Y are independent random variables I am finishing my statistics course at school and this week, we learned how to estimate a random variable X to minimize MSE.
One of the cases is to estimate X given another random variable Y. I was told that the MMSE estimator for X given Y is
$$\hat{X}_M(Y)= E[X|Y]$$
However, the MMSE estimator, most of the time, is hard to compute. Instead, it is easier to compute the LMSE estimator for X given Y:
$$\hat{X}_L(Y)= aY + b$$
Here is my question. What if X and Y are independent random variables? If they are independent, then $f_{X|Y}(x|y) = f_X(x)$. Therefore,
$$E[X|Y] = \int_{-\infty}^{\infty} xf_{X|Y}(x|y)dx = \int_{-\infty}^{\infty} xf_{X}(x)dx = E[X]$$
(in case X is continuous)
So to estimate X given Y when X and Y are independent is the same as to do a blind estimation of X. Am I correct? What about the LMSE estimator?
Thank you all!
 A: As lafinur noted in the comments, you are correct. $X$ and $Y$ being independent means that they don’t contain any information about each other, so knowing $Y$ doesn’t change how you can estimate $X$.
To show this concretely for the LMSE , minimize the mean square error:
\begin{eqnarray*}
\def\E{\mathsf E}
\frac{\partial}{\partial a}\E\left[\left(\hat X_L(Y)-X\right)^2\right]
&=&
\frac{\partial}{\partial a}\E\left[\left(aY+b-X\right)^2\right]
\\
&=&
2E\left[Y(aY+b-X)\right]\\
&=&
2\left(a\E\left[Y^2\right]+b\E[Y]-\E[XY]\right)\\
&=&
2\left(a\E\left[Y^2\right]+b\E[Y]-\E[X]\E[Y]\right)\\
&\stackrel!=&0
\end{eqnarray*}
and
\begin{eqnarray*}
\frac{\partial}{\partial b}\E\left[\left(\hat X_L(Y)-X\right)^2\right]
&=&
\frac{\partial}{\partial b}\E\left[\left(aY+b-X\right)^2\right]
\\
&=&
2\E\left[aY+b-X\right]\\
&=&
2\left(a\E\left[Y^2\right]+b\E[Y]-\E[XY]\right)\\
&=&
2\left(a\E\left[Y\right]+b-\E[X]\right)\\
&\stackrel!=&0\;.
\end{eqnarray*}
Subtract $\E[Y]$ times the second equation from the first to show that $a=0$ if $\mathsf{Var}[Y]=\E\left[Y^2\right]-\E[Y]^2\ne0$, and then solve the second equation to obtain $b=\E[X]$. Thus, in this case the LMSE is just $\hat X_L(Y)=\E[X]$.
If $\mathsf{Var}[Y]=0$ (i.e. if $Y$ is almost surely constant), the two equations are linearly dependent, and any estimator of the form $\hat X_L(Y)=aY+\E[X]-a\E[Y]$ for any $a$ will do (since $Y$ is almost surely equal to $\E[Y]$).
