W and b for LMMSE using covariance(XY)

Question

I would like to calculate the $W_{LMMSE}$ and $b_{LMMSE}$ for X which is a uniform random variable between $-\pi/2$ and $\pi/2$ and $Y=\sin(X)$. I have the following info:

$\Sigma_{XY} = 2/\pi$

$\Sigma_X = \pi^2/12$

$\mu_X=0$

I want to confirm if the following formula are correct?

$$W_{OLS} = \Sigma_{XY}/\Sigma_{X} = 24/\pi^3$$

and

$$b_{OLS} = \Sigma_{XY} - W_{OLS} \cdot \mu_X = 2/\pi$$

We have

$$(W_{LMMSE}, b_{LMMSE}) = \text{argmin}_{W, b} E[(Y-WX-b)^2] $$

I assume in this case Ordinary Least Square is same as Linear MMSE (Minimum Mean Squared Error)(LMMSE) (please correct me if I am wrong).

Answer 1

Your second formula should be $$ b_{OLS} = \mu_Y - W_{OLS}\cdot \mu_X = 0. $$ Indeed, let us find $(W_{LMMSE}, b_{LMMSE}) = \text{argmin}_{W, b} \mathop{\mathbb E}[(Y-WX-b)^2]$ by blunt frontal calculation. $$ \mathop{\mathbb E}[(Y-WX-b)^2]=\mathop{\mathbb E}[Y^2]+W^2\mathop{\mathbb E}[X^2]+b^2-2W\mathop{\mathbb E}[XY]+2bW\mathop{\mathbb E}[X]-2b\mathop{\mathbb E}[Y]. $$ Find the partial derivatives w.r.t $W$ and $b$ and equate them to zero: $$ 2W\mathop{\mathbb E}[X^2]-2\mathop{\mathbb E}[XY]+2b\mathop{\mathbb E}[X]=0 $$ $$ 2b+2W\mathop{\mathbb E}[X]-2\mathop{\mathbb E}[Y]=0 $$ Find $b=\mu_Y-W\cdot\mu_X$ from the second equation and substitute it to the first to find $W$. We get finally $$ W_{OLS} = \frac{\mathop{\mathbb E}[XY] - \mathop{\mathbb E}[X]\mathop{\mathbb E}[Y]}{\mathop{\mathbb E}[X^2]-(\mathop{\mathbb E}[X])^2}=\frac{\Sigma_{XY}}{\Sigma_X},\quad b_{OLS}=\mu_Y - W_{OLS}\cdot\mu_X. $$

You can find these forumula in Wiki: https://en.wikipedia.org/wiki/Minimum_mean_square_error#Linear_MMSE_estimator