# Random sampling, with and without replacement: difference between MSEs?

In statistics, a simple random sample is a subset of individuals chosen (one by one) from a population. Each individual is chosen randomly such that each individual has the same probability of being chosen at any stage during the sampling process, and each subset of $$k$$ individuals has the same probability of being chosen for the sample as any other subset of $$k$$ individuals. From a population of size $$N$$ with finite variance, a simple random sample of size $$n$$ is drawn without replacement, and a real-valued characteristic $$X$$ measured to yield observation $$X_j$$ $$(j = 1,2,3, \ldots,n)$$.

(a) show that the sample mean $$\overline X_n$$ is an unbiased estimator of the population mean $$m$$.

(b) show that the expected squared error of $$\overline X_n$$ as an estimator of $$m$$ is smaller than that of the mean of a simple random sample of the same size $$n$$ drawn with replacement.

(c) show that as $$n,N \to \infty$$ and $$r=\frac{n}{N}$$ and the population variance is always less than $$M$$ for all $$N$$, the difference between the expected squared errors of the two estimators is $$O(r)$$

For the question (b), I worked out the expected squared error of $$\overline X_n$$ is $$\frac{N-n}{N-1}\frac{\sigma^2}{n}$$. The other estimator with replacement is $$\frac{\sigma^2}{n}$$.

As to (c), the difference between them is $$O\left(\frac{1}{N}\right)$$. I wonder where I make the mistake? Thanks a lot.

• Welcome to MSE. It is in your best interest that you type your questions (using MathJax) instead of posting links to pictures. – José Carlos Santos Sep 10 at 8:47

Your formulae are correct, the difference between the two is $$\frac{\sigma^2}{rN}\left(\frac{N-rN}{N-1} - 1\right) \sim_{n,N\to\infty} -\frac{\sigma^2}{N}$$
The relative error expressed in % is of order $$r$$ though : $$\frac{\frac{\sigma^2}{rN}\left(\frac{N-rN}{N-1} - 1\right)}{\frac{\sigma^2}{rN}} = \frac{1-rN}{N-1}\sim_{n,N\to \infty} -r$$
• Maybe it's a typo from the professor, but maybe he really meant $O(r)$... The best solution is to ask him at the end of the course if you see him before giving in, or by email otherwise. – thomasb Sep 10 at 12:53