Finding variance of the sample mean of a random sample of size n without replacement from finite population of size N.

Question

I encountered this problem in the book "Introduction to the Theory of Statistics" (by Mood, Graybill and Boes) and I have not been able to solve part (c):

"A bowl contains five chips numbered from 1 to 5. A sample of two drawn without replacement from this finite population is said to be random if all possible pairs of the five chips have an equal chance to be drawn.
(a) What is the expected value of the sample mean? What is the variance of the sample mean?
(b) Suppose that the two chips of part (a) were drawn with replacement. What would be the variance of the sample mean? Why might one guess that this variance would be larger than the one obtained before?
(c) Generalize part (a) by considering N chips and samples of size n. Show that the variance of the sample mean is $$\frac{N-n}{N-1}\frac{\sigma^{2}}{n}$$ where $\sigma^{2}$ is the population variance, that is $$\sigma^{2}=\frac{1}{N}\sum_{i=1}^{N}\Big(i-\frac{N+1}{2}\Big)^{2}$$"
* To solve part (a) I explicitly wrote the set of possible pairs with equal probability: $\Omega=\{(1,2),(1,3),(1,4),(1,5),(2,3),(2,4),(2,5),(3,4),(3,5),(4,5)\}$
From this it is easy to see that $Im\bar{X}=\{1.5,2,2.5,3,3.5,4,4.5\}$ where $\bar{X}=\frac{1}{n}\sum_{i=1}^{n} X_{i}$
Correspondingly the probabilities for this values are $(0.1,0.1,0.2,0.2,0.2,0.1,0.1)$
Hence, by definition, the expected value and the variance are: $E[\bar{X}]=3$ and $V[\bar{X}]=\frac{3}{4}$.
* For part (b) the same procedure gives us $E[\bar{X}]=3$ and $V[\bar{X}]=\frac{7}{6}$.
* Finally, for part (c) I tried to generalize what I did noticing that the least value for $\sum X_{i}$ is $\frac{n(n+1)}{2}$ and its greatest possible value is $\frac{n(2N-n+1)}{2}$.
Hence $Im\bar{X}=\{\frac{n+1}{2},\frac{n+1}{2}+\frac{1}{n},\frac{n+1}{2}+\frac{2}{n},\dots,\frac{n+1}{2}+(N-n)\}$ Clearly the probability for the first and last values is $\frac{1}{C^{N}_{n}}=\frac{n!(N-n)!}{N!}$ but I haven't come up with an idea of how to find the other probabilities. How can I get the rest of them?

Answer 1

It is time to upgrade the methods you rely on, since listing every possibility yields the result for small values of $n$ and $N$ but this approach (1) leads to a dead end for general values (as you realized), and (2) provides no insight.

So... let us attack directly (c), considering $N$ chips numbered from $1$ to $N$, and samples of size $n$. Then the sample mean $\bar X$ is such that $n\bar X=\sum\limits_{k=1}^nY_k$ where $Y_k$ is the $k$th chip. By hypothesis, each $Y_k$ is uniform on $\{1,2,\ldots,N\}$ hence $E(Y_k)=\frac1N\sum\limits_{x=1}^Nx=\frac12(N+1)$ for every $k$ and, by linearity of the expectation, $$E(\bar X)=\frac{N+1}2.$$ Likewise, $n^2\bar X^2=\sum\limits_{k=1}^nY_k^2+\sum\limits_{k\ne\ell}Y_kY_\ell$, $E(Y_k^2)$ does not depend on $k$ and $E(Y_kY_\ell)$ does not depend on $k\ne\ell$, hence $nE(\bar X^2)=E(Y_1^2)+(n-1)E(Y_1Y_2)$ hence it suffices to compute $E(Y_1^2)$ and $E(Y_1Y_2)$. By the same argument as before, $E(Y_1^2)=\frac1N\sum\limits_{x=1}^Nx^2=\frac16(N+1)(2N+1)$. To compute $E(Y_1Y_2)$, one can note that, conditionally on $Y_k=x$, $Y_\ell$ is independent on $\{1,2,\ldots,N\}\setminus\{x\}$ and proceed.

Or, one can use the specific case $n=N$, since then, $N\bar X=\sum\limits_{x=1}^Nx$ with full probability, in particular, $NE(Y_1^2)+N(N-1)E(Y_1Y_2)=\frac14N^2(N+1)^2$, which leads to $E(Y_1Y_2)=\frac1{12}(N+1)(3N+2)$. Finally, $$E(\bar X^2)=\frac{2(N+1)(2N+1)+(N+1)(3N+2)(n-1)}{12n},$$ and the variance follows.

Sanity check: If $n=N$, then $E(\bar X^2)=E(\bar X)^2$ (do you see why?).