# $E(Z_iZ_j) = E(Z_i)E(Z_j)$ for independent r.v.s

Consider a population $$\omega_1, \omega_2, ..., \omega_N$$ of $$N$$ individuals. Choose, with replacement, a sample of $$n$$ individuals from the population, denoting this outcome by the ordered list $$(\zeta_{1}, \zeta_{2}, ..., \zeta_{N})$$. Let $$X$$ be a random variable. Denote $$Z_i = X(\zeta_{i})$$ the value of $$X$$ for the $$i^{\text{th}}$$ member of the sample.

I am trying to show that the covariance $$\text{cov}(Z_i,Z_j)$$ of r.v.s $$Z_i, Z_j, i \neq j,$$ is zero, by showing that $$\mathbb{E}(Z_iZ_j) = \mathbb{E}(Z_i)\mathbb{E}(Z_j)$$. I get the following $$\text{cov}(Z_i, Z_j) = \mathbb{E}(Z_iZ_j) - \mathbb{E}(Z_i)\mathbb{E}(Z_j),$$ where $$\mathbb{E}(Z_i) = \frac{1}{N}(X_1 + \cdots + X_n) = \bar{X}$$ and \begin{align} \mathbb{E}(Z_iZ_j) &= \frac{1}{N^2}\sum_{k \neq \ell} x_{k, \ell} \\ &= \frac{1}{N^2} \left(\left(\sum_{i=1}^{N} x_i\right)^2 - \sum_{i=1}^{N} x_i^2 \right) \\ &= \frac{1}{N^2}\left(N^2\bar{X}^2 - \sum_{i=1}^{N} x_i^2\right). \end{align}

It can be. clearly seen from here that my $$\text{cov}(Z_i, Z_j) \neq 0$$. How can I remedy this?

EDIT: I have emphasised with replacement, thereby implying that $$Z_1, Z_2, ..., Z_n$$ are independent.

• You seem to be confusing the mean with the (empirical) sample mean. Sep 29, 2023 at 14:41
• @Aruralreader I wish to prove this for the empirical sample variance. Sep 29, 2023 at 14:41
• Thanks for clarifying that you're doing "with replacement". In that case, removing the $k=l$ case is not necessary. Sep 29, 2023 at 15:47
• Also, it is confusing to say that "$X$ is a random variable". In my understanding, for each $k\in\{1,\dots,N\}$, the quantity $X_k$ is already fixed. Isn't this right? Sep 29, 2023 at 16:07
• @BenjaminWang I am not sure, I am following a notation given in class. All I know is that $(\xi_1, ..., x_n)$ is a random sample, on which we apply the random variable $X$. Sep 29, 2023 at 17:38

Let $$X_k = X(\omega_k)$$ for $$k\in\{1,\dots,N\}$$. Let $$I_{ik} = \mathbb{1}(\zeta_i = \omega_k) = \mathbb{1}($$the $$i$$th sample is the $$k$$th individual in the population).

The randomness in this system is your method of choosing $$Z_i,Z_j$$.

Note that $$Z_i = \sum_{k=1}^N X_kI_{ik}$$. So

\begin{align} E(Z_iZ_j) &= E\left(\left(\sum_{k=1}^N X_kI_{ik}\right) \cdot\left(\sum_{l=1}^N X_lI_{jl}\right)\right)\\ &=\sum_{k=1}^N\sum_{l=1}^N X_kX_l E(I_{ik}I_{jl})\\ &=\frac{1}{N^2}\sum_{k=1}^N\sum_{l=1}^N X_kX_l \end{align}

Note that there's no need to subtract $$X_kX_k$$: just because $$k=l$$ doesn't mean we can't have $$I_{ik}=I_{jl}=1$$; remember we are choosing with replacement.

We also have $$E(Z_i) = E\left(\sum_{k=1}^N X_kI_{ik}\right) = \frac1N \sum_{k=1}^N X_k$$, so $$E(Z_i)E(Z_j) = E(Z_iZ_j)$$.

• It might be worth noting that, in terms of the (real) random variable $X$, your $X_k=X(\omega_k).$ Also, in terms of the (not necessarily real) random variables $\zeta_i,$ your $I_{ik}=1_{\zeta_i=\omega_k}.$ Oct 1, 2023 at 17:16
• @r.e.s. Thank you. I've fixed it. Oct 1, 2023 at 19:17