What is the expected value of a Bernouilli matrix with Hadamard product In this thesis on page 18, it says these two equations are equivalent:
$$
\mathbb{E}_{R \sim \text{Bernouilli}(p)} \left[|| \mathbf{y} - (R*X)\mathbf{w}||^2 \right] 
$$
$$
||\mathbf{y} - pX\mathbf{w}||^2 + p(1-p)||\Gamma \mathbf{w}||^2
$$
where $\Gamma = (\text{diag}(X^TX))^{1/2}$, $\mathbf{y},\mathbf{w}$ are vectors, $R,X$ are matrices, $p$ is a scalar, $*$ is element-wise multiplication, and 2 i the number 2.
How do you get from equation 1 to 2? I'm confused since the expected value of a Bernouilli is $p$, so why the second part in the second equation?
 A: First, let's develop the mean:
$$\text{E}_R \left[ \left\lVert y -(R*X) w\right\rVert^2 \right]= \text{E}_R \left[(y -(R*X) w)^T(y -(R*X) w)\right]=\dots $$ 
$$ \dots = y^Ty - 2 y^T \text{E}_R[R*X]w+w^T\text{E}_R[(R*X)^T(R*X)]w \quad \quad \quad \textbf{Result 1}$$
Now, we need to compute terms $\text{E}_R[R*X]$ and $\text{E}_R[(R*X)^T(R*X)]$. First term is immediate:
$$\text{E}_R[(R*X)]_{i,j} = pX_{i,j} \quad \quad \quad \textbf{Result 2}$$
While for the second term, we need to differentiate two cases:


*

*$i \neq j$ (out off diagonal terms):
$R_{k,i}$ and $R_{k,j}$ are independent, so $\text{E}_R[R_{k,i} R_{k,j}]=\text{E}_R[R_{k,i}]\text{E}_R[ R_{k,j}]=p^2$:
$$\text{E}_R[(R*X)^T(R*X)]_{i,j} = \sum_{k=1}^N(X_{k,i}\text{E}_R[R_{k,i}]\text{E}_R[ R_{k,j}]X_{k,j}) = p^2 (X^T X)_{i,j}$$

*$i = j$ (diagonal terms):
$R_{k,i}$ and $R_{k,j}$ are dependent (they are the same term), so one an prove that $\text{E}_R[R_{k,i} R_{k,j}]= p$ (see here):
$$\text{E}_R[(R*X)^T(R*X)]_{i,j} = p(X^T X)_{i,j}$$
Combining both cases, one can write a general expression such as:
$$\text{E}_R[(R*X)^T(R*X)]_{i,j} = p^2 (X^T X)_{i,j}(1- \delta_{i,j})+p(X^T X)_{i,j} \delta_{i,j}$$
And removing indexes:
$$\text{E}_R[(R*X)^T(R*X)] = p^2 (X^T X)+p(1-p)\text{diag}(X^T X) \quad \quad \quad \textbf{Result 3}$$
Finally, combining Result 1, Result 2 and Result 3, leads to:
$$\text{E}_R \left[ \left\lVert y -(R*X) w\right\rVert^2 \right]= \left\lVert y-pXw\right\rVert^2+p(1-p)\left\lVert \text{diag}(X^T X)^{1/2}w\right\rVert^2$$
Proving the desired expression.
