# Variance Estimator in Simple Random Sampling Without Replacement

I have to find the unbiased estimator of population variance under simple random sampling without replacement.

The hint for the demonstration is: $$\frac{1}{N} \sum_{k =1}^{N} (x_{k} - \bar{x_{U}})^2 = \frac{1}{2N^2} \sum_{k =1}^{N}\sum_\underset{\Large{l\neq k}}{l=1}^{N} (x_{k} - \bar{x_{l}})^2$$

I start like this, but I don't know if this is right:

$$\implies \frac{1}{2N^2}\sum_\underset{\Large{l\neq k}}{l=1}^{N} (x_{k}- \bar{x_{l}})^2$$

$$\implies \frac{1}{2N^2}\sum_\underset{\Large{l\neq k}}{l=1}^{N} (x_{k}^2 - 2x_{l}x_{k} + \bar{x_{l}}^2)$$

$$\implies \frac{1}{2N^2}\sum_\underset{\Large{l\neq k}}{l=1}^{N} x_{k}^2 -\sum_\underset{\Large{l\neq k}}{l=1}^{N}2\bar{x_{l}}x_{k} +\sum_\underset{\Large{l\neq k}}{l=1}^{N}\bar{x_{l}}^2$$

Here I am stuck.

• What is $x_k,x_U$? Have you taken any expectations? Commented Jul 21, 2020 at 7:10
• I have clear that $x_{U}$ is the population mean. But I think that $x_{k}$ is my sample observation Commented Jul 21, 2020 at 11:48
• The hint is an identity for any list of numbers $(x_1,...,x_N)$, where $x_U:={1\over N}\sum_{i=1}^N;$ so it holds regardless of whether $(x_1,...,x_N)$ is a population or a sample. (If $(x_1,...,x_N)$ is a sample, then the identity won't hold unless $x_U$ is the sample mean; similarly for population and population mean.) You can apply this directly to the definition of the sample variance of sample $(y_1,...,y_n)$, so its expectation involves $E(y_{k} - y_{l})^2=E(y_1-y_2)^2= 2\left(\sigma^2-\text{cov}(y_1,y_2)\right)$, where $\sigma^2$ is the population variance, etc. Commented Jul 21, 2020 at 15:11
• My comment above refers to the identity as originally posted (proved here). But after your recent edit, which added bars in the RHS, that equation now seems unintelligible. (The original identity can be used in the way I described to derive an unbiased estimator, but getting the covariance term is tedious.) Commented Jul 21, 2020 at 21:38
• I found a way to get the covariance term easily, and posted it in my answer. Commented Jul 21, 2020 at 23:29

Let $$(y_1,...,y_n)$$ be a simple random sample without replacement from population $$(x_1,...,x_N).$$ Then the population mean and variance are, respectively, \begin{align}\mu:&={1\over N}\sum_{i=1}^Nx_i\\ \sigma^2:&={1\over N}\sum_{i=1}^N(x_i-\mu)^2.\end{align} Following is a sketch of how to show that \begin{align}E\left({N-1\over N}{1\over n-1}\sum_{i=1}^n(y_i-\bar{y})^2\right)=\sigma^2.\end{align}

Aside: Some authors differ on the definition of "population variance", taking it to be the quantity $$S^2:={N\over N-1}\sigma^2= {1\over N-1}\sum_{i=1}^N(x_i-\mu)^2,$$ presumably to allow the above unbiasedness result to be written as follows:

\begin{align}E\left({1\over n-1}\sum_{i=1}^n(y_i-\bar{y})^2\right)=S^2.\end{align}

By the OP's identity (as originally posted, which is proved here),

\begin{align}E\left(\frac{1}{n} \sum_{i =1}^{n} (y_{i} - \bar{y})^2\right) &= \frac{1}{2n^2} \sum_{i =1}^{n}\sum_\underset{\Large{j\neq i}}{j=1}^{n} E(y_i - y_j)^2\\ &={1\over 2n^2} n(n-1)E(y_1-y_2)^2\\ &={1\over 2n^2} n(n-1)E\left((y_1-\mu)-(y_2-\mu)\right)^2\\ &={1\over 2n^2} n(n-1)E\left((y_1-\mu)^2+(y_2-\mu)^2-2(y_1-\mu)(y_2-\mu)\right)\\ &={1\over 2n^2} n(n-1)\,2(\sigma^2-\text{cov}(y_1,y_2))\\ &={1\over 2n^2} n(n-1)\,2(\sigma^2-(-{\sigma^2\over N-1}))\\[2ex] &={n-1\over n}{N\over N-1}\sigma^2. \quad\quad\quad\quad\quad\quad\quad\quad\text{QED}\end{align} In the above, the covariance term is obtained as follows, because each of the $$N(N-1)$$ possible outcomes for $$(y_1-\mu)(y_2-\mu)$$ is equally likely: \begin{align}\text{cov}(y_1,y_2) &=E\left((y_1-\mu)(y_2-\mu)\right)\\ &=\frac{1}{N(N-1)} \sum_{i =1}^{N}\sum_\underset{\Large{j\neq i}}{j=1}^{N} (x_i-\mu)(x_j-\mu)\\ &=\frac{1}{N(N-1)} (-N\sigma^2)\\ &=-{\sigma^2\over N-1} \end{align} where we have used $$\sum_{i =1}^{N}\sum_\underset{\Large{j\neq i}}{j=1}^{N} (x_i-\mu)(x_j-\mu)=-N\sigma^2$$ which is a consequence of the following identity: \begin{align}0^2=\left(\sum_{i=1}^N(x_i-\mu)\right)^2 &=\sum_{i=1}^N(x_i-\mu)^2 + \sum_{i =1}^{N}\sum_\underset{\Large{j\neq i}}{j=1}^{N} (x_i-\mu)(x_j-\mu)\tag{*}\\ &=N\sigma^2 + \sum_{i =1}^{N}\sum_\underset{\Large{j\neq i}}{j=1}^{N} (x_i-\mu)(x_j-\mu).\end{align}

Note that (*) is just a special case (with $$z_i=x_i-\mu$$, so $$\sum z_i=0$$) of the general identity $$\left(\sum_{i=1}^N z_i\right)^2 =\sum_{i=1}^Nz_i^2 + \sum_{i =1}^{N}\sum_\underset{\Large{j\neq i}}{j=1}^{N}z_iz_j.$$

Sources: