Consider a population $\omega_1, \omega_2, ..., \omega_N$ of $N$ individuals. Choose, with replacement, a sample of $n$ individuals from the population, denoting this outcome by the ordered list $(\zeta_{1}, \zeta_{2}, ..., \zeta_{N})$. Let $X$ be a random variable. Denote $Z_i = X(\zeta_{i})$ the value of $X$ for the $i^{\text{th}}$ member of the sample.

I am trying to show that the covariance $\text{cov}(Z_i,Z_j)$ of r.v.s $Z_i, Z_j, i \neq j,$ is zero, by showing that $\mathbb{E}(Z_iZ_j) = \mathbb{E}(Z_i)\mathbb{E}(Z_j)$. I get the following $$\text{cov}(Z_i, Z_j) = \mathbb{E}(Z_iZ_j) - \mathbb{E}(Z_i)\mathbb{E}(Z_j),$$ where $\mathbb{E}(Z_i) = \frac{1}{N}(X_1 + \cdots + X_n) = \bar{X}$ and \begin{align} \mathbb{E}(Z_iZ_j) &= \frac{1}{N^2}\sum_{k \neq \ell} x_{k, \ell} \\ &= \frac{1}{N^2} \left(\left(\sum_{i=1}^{N} x_i\right)^2 - \sum_{i=1}^{N} x_i^2 \right) \\ &= \frac{1}{N^2}\left(N^2\bar{X}^2 - \sum_{i=1}^{N} x_i^2\right). \end{align}

It can be. clearly seen from here that my $\text{cov}(Z_i, Z_j) \neq 0$. How can I remedy this?

EDIT: I have emphasised with replacement, thereby implying that $Z_1, Z_2, ..., Z_n$ are independent.

  • $\begingroup$ You seem to be confusing the mean with the (empirical) sample mean. $\endgroup$ Sep 29, 2023 at 14:41
  • $\begingroup$ @Aruralreader I wish to prove this for the empirical sample variance. $\endgroup$ Sep 29, 2023 at 14:41
  • 1
    $\begingroup$ Thanks for clarifying that you're doing "with replacement". In that case, removing the $k=l$ case is not necessary. $\endgroup$ Sep 29, 2023 at 15:47
  • $\begingroup$ Also, it is confusing to say that "$X$ is a random variable". In my understanding, for each $k\in\{1,\dots,N\}$, the quantity $X_k$ is already fixed. Isn't this right? $\endgroup$ Sep 29, 2023 at 16:07
  • $\begingroup$ @BenjaminWang I am not sure, I am following a notation given in class. All I know is that $(\xi_1, ..., x_n)$ is a random sample, on which we apply the random variable $X$. $\endgroup$ Sep 29, 2023 at 17:38

1 Answer 1


Let $X_k = X(\omega_k)$ for $k\in\{1,\dots,N\}$. Let $I_{ik} = \mathbb{1}(\zeta_i = \omega_k) = \mathbb{1}($the $i$th sample is the $k$th individual in the population).

The randomness in this system is your method of choosing $Z_i,Z_j$.

Note that $Z_i = \sum_{k=1}^N X_kI_{ik}$. So

$$ \begin{align} E(Z_iZ_j) &= E\left(\left(\sum_{k=1}^N X_kI_{ik}\right) \cdot\left(\sum_{l=1}^N X_lI_{jl}\right)\right)\\ &=\sum_{k=1}^N\sum_{l=1}^N X_kX_l E(I_{ik}I_{jl})\\ &=\frac{1}{N^2}\sum_{k=1}^N\sum_{l=1}^N X_kX_l \end{align} $$

Note that there's no need to subtract $X_kX_k$: just because $k=l$ doesn't mean we can't have $I_{ik}=I_{jl}=1$; remember we are choosing with replacement.

We also have $E(Z_i) = E\left(\sum_{k=1}^N X_kI_{ik}\right) = \frac1N \sum_{k=1}^N X_k$, so $E(Z_i)E(Z_j) = E(Z_iZ_j)$.

  • $\begingroup$ It might be worth noting that, in terms of the (real) random variable $X$, your $X_k=X(\omega_k).$ Also, in terms of the (not necessarily real) random variables $\zeta_i,$ your $I_{ik}=1_{\zeta_i=\omega_k}.$ $\endgroup$
    – r.e.s.
    Oct 1, 2023 at 17:16
  • $\begingroup$ @r.e.s. Thank you. I've fixed it. $\endgroup$ Oct 1, 2023 at 19:17

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .