0
$\begingroup$

I have to find the unbiased estimator of population variance under simple random sampling without replacement.

The hint for the demonstration is: $$ \frac{1}{N} \sum_{k =1}^{N} (x_{k} - \bar{x_{U}})^2 = \frac{1}{2N^2} \sum_{k =1}^{N}\sum_\underset{\Large{l\neq k}}{l=1}^{N} (x_{k} - \bar{x_{l}})^2 $$

I start like this, but I don't know if this is right:

$\implies \frac{1}{2N^2}\sum_\underset{\Large{l\neq k}}{l=1}^{N} (x_{k}- \bar{x_{l}})^2$

$\implies \frac{1}{2N^2}\sum_\underset{\Large{l\neq k}}{l=1}^{N} (x_{k}^2 - 2x_{l}x_{k} + \bar{x_{l}}^2)$

$\implies \frac{1}{2N^2}\sum_\underset{\Large{l\neq k}}{l=1}^{N} x_{k}^2 -\sum_\underset{\Large{l\neq k}}{l=1}^{N}2\bar{x_{l}}x_{k} +\sum_\underset{\Large{l\neq k}}{l=1}^{N}\bar{x_{l}}^2$

Here I am stuck.

$\endgroup$
5
  • $\begingroup$ What is $x_k,x_U$? Have you taken any expectations? $\endgroup$ Jul 21, 2020 at 7:10
  • $\begingroup$ I have clear that $x_{U}$ is the population mean. But I think that $x_{k}$ is my sample observation $\endgroup$ Jul 21, 2020 at 11:48
  • $\begingroup$ The hint is an identity for any list of numbers $(x_1,...,x_N)$, where $x_U:={1\over N}\sum_{i=1}^N;$ so it holds regardless of whether $(x_1,...,x_N)$ is a population or a sample. (If $(x_1,...,x_N)$ is a sample, then the identity won't hold unless $x_U$ is the sample mean; similarly for population and population mean.) You can apply this directly to the definition of the sample variance of sample $(y_1,...,y_n)$, so its expectation involves $E(y_{k} - y_{l})^2=E(y_1-y_2)^2= 2\left(\sigma^2-\text{cov}(y_1,y_2)\right)$, where $\sigma^2$ is the population variance, etc. $\endgroup$
    – r.e.s.
    Jul 21, 2020 at 15:11
  • $\begingroup$ My comment above refers to the identity as originally posted (proved here). But after your recent edit, which added bars in the RHS, that equation now seems unintelligible. (The original identity can be used in the way I described to derive an unbiased estimator, but getting the covariance term is tedious.) $\endgroup$
    – r.e.s.
    Jul 21, 2020 at 21:38
  • $\begingroup$ I found a way to get the covariance term easily, and posted it in my answer. $\endgroup$
    – r.e.s.
    Jul 21, 2020 at 23:29

1 Answer 1

2
$\begingroup$

Let $(y_1,...,y_n)$ be a simple random sample without replacement from population $(x_1,...,x_N).$ Then the population mean and variance are, respectively, $$\begin{align}\mu:&={1\over N}\sum_{i=1}^Nx_i\\ \sigma^2:&={1\over N}\sum_{i=1}^N(x_i-\mu)^2.\end{align}$$ Following is a sketch of how to show that $$\begin{align}E\left({N-1\over N}{1\over n-1}\sum_{i=1}^n(y_i-\bar{y})^2\right)=\sigma^2.\end{align}$$


Aside: Some authors differ on the definition of "population variance", taking it to be the quantity $$S^2:={N\over N-1}\sigma^2= {1\over N-1}\sum_{i=1}^N(x_i-\mu)^2,$$ presumably to allow the above unbiasedness result to be written as follows:

$$\begin{align}E\left({1\over n-1}\sum_{i=1}^n(y_i-\bar{y})^2\right)=S^2.\end{align}$$


By the OP's identity (as originally posted, which is proved here),

$$\begin{align}E\left(\frac{1}{n} \sum_{i =1}^{n} (y_{i} - \bar{y})^2\right) &= \frac{1}{2n^2} \sum_{i =1}^{n}\sum_\underset{\Large{j\neq i}}{j=1}^{n} E(y_i - y_j)^2\\ &={1\over 2n^2} n(n-1)E(y_1-y_2)^2\\ &={1\over 2n^2} n(n-1)E\left((y_1-\mu)-(y_2-\mu)\right)^2\\ &={1\over 2n^2} n(n-1)E\left((y_1-\mu)^2+(y_2-\mu)^2-2(y_1-\mu)(y_2-\mu)\right)\\ &={1\over 2n^2} n(n-1)\,2(\sigma^2-\text{cov}(y_1,y_2))\\ &={1\over 2n^2} n(n-1)\,2(\sigma^2-(-{\sigma^2\over N-1}))\\[2ex] &={n-1\over n}{N\over N-1}\sigma^2. \quad\quad\quad\quad\quad\quad\quad\quad\text{QED}\end{align}$$ In the above, the covariance term is obtained as follows, because each of the $N(N-1)$ possible outcomes for $(y_1-\mu)(y_2-\mu)$ is equally likely: $$\begin{align}\text{cov}(y_1,y_2) &=E\left((y_1-\mu)(y_2-\mu)\right)\\ &=\frac{1}{N(N-1)} \sum_{i =1}^{N}\sum_\underset{\Large{j\neq i}}{j=1}^{N} (x_i-\mu)(x_j-\mu)\\ &=\frac{1}{N(N-1)} (-N\sigma^2)\\ &=-{\sigma^2\over N-1} \end{align}$$ where we have used $$\sum_{i =1}^{N}\sum_\underset{\Large{j\neq i}}{j=1}^{N} (x_i-\mu)(x_j-\mu)=-N\sigma^2$$ which is a consequence of the following identity: $$\begin{align}0^2=\left(\sum_{i=1}^N(x_i-\mu)\right)^2 &=\sum_{i=1}^N(x_i-\mu)^2 + \sum_{i =1}^{N}\sum_\underset{\Large{j\neq i}}{j=1}^{N} (x_i-\mu)(x_j-\mu)\tag{*}\\ &=N\sigma^2 + \sum_{i =1}^{N}\sum_\underset{\Large{j\neq i}}{j=1}^{N} (x_i-\mu)(x_j-\mu).\end{align}$$

Note that (*) is just a special case (with $z_i=x_i-\mu$, so $\sum z_i=0$) of the general identity $$\left(\sum_{i=1}^N z_i\right)^2 =\sum_{i=1}^Nz_i^2 + \sum_{i =1}^{N}\sum_\underset{\Large{j\neq i}}{j=1}^{N}z_iz_j. $$

Sources:

http://dept.stat.lsa.umich.edu/~moulib/sampling.pdf https://issuu.com/patrickho77/docs/mth_432a_-_introduction_to_sampling

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .