What Laws of Mathematics Best Explain How Surveys Work? Suppose I have some probability distribution - as an example, I choose the Normal Distribution with some very large variance.
Suppose I assume that the distribution of the number of calories eaten a day follows such a Normal Distribution in a country (population = 1000000 people) with a very large population. Now, let's say that I can only ask a very small percentage (50 people) of these people how many calories they consume every day, and I am interested in estimating the true number of calories the average person eats in one day within the population.
Here is some R code to simulate this:
# lets assume that the true average is 2000 calories, but this is unknown
set.seed(123)
population_calories = rnorm(1000000, 2000, 1000)

Suppose a researcher randomly selects 50 people from this country and asks how many calories   they eat and takes the average:
mean(sample(population_calories , 50, replace=FALSE))
[1] 1988.098

As we see, this number is very close to the actual average.
Now, if we repeat this for 100 researchers:
my_list = list()

for (i in 1:100)

{
sample_i = mean(sample(population_calories , 50, replace=FALSE))
my_list[[i]] = data.frame(i,sample_i)
}

Looking at the distribution of these results:
m = do.call(rbind.data.frame, my_list)
plot(density(m$sample_i))
mean(m$sample_i)
[1] 1999.711


We see that the average estimates from a very small sample (0.005% of the population) from a  population with considerably large variance comes very close to the true value!
Furthermore, I have heard that even when the underlying distribution is not a Normal Distribution, the above phenomena would still repeat.
I was wondering - what principles of mathematics best explain this above phenomena?
Is this more an application of the "Central Limit Theorem" (https://en.wikipedia.org/wiki/Central_limit_theorem) or "Weak Law of Large Numbers" (https://en.wikipedia.org/wiki/Law_of_large_numbers)?
Thanks!
 A: The central limit theorem and the law of large numbers are both conceptually relevant but neither, strictly speaking, explains this result, because they both only concern what happens in the limit when the number of random samples from the population approaches $\infty$, whereas for applications to surveys we of course want to understand what happens for a fixed finite number $n$ of samples.
The simplest calculation relevant to this question is the following. Let $X_1, X_2, \dots$ be a sequence of independent and identically distributed random variables with common distribution $X$, to be interpreted as random samples of some statistic of interest from a large population. We want to understand the distribution of the sequence of sample means
$$S_n = \frac{X_1 + X_2 + \dots + X_n}{n}.$$
If $\mu = \mathbb{E}(X)$ is the true mean of the random variable from which we are sampling then $\mathbb{E}(S_n) = \mu$ also has expected value $\mu$. The law of large numbers tells us that, in a suitable sense, $S_n$ converges to $\mu$ as $n \to \infty$. But what happens for a specific finite value of $n$, say $n = 100$?
To get a first handle on this we can compute the variance of the sample mean. If we write $\sigma^2 = \text{Var}(X)$ for the true variance then it's a good exercise to check that
$$\text{Var}(S_n) = \frac{\sigma^2}{n}.$$
That is, averaging $n$ iid samples has the effect of reducing the variance by a factor of $n$, or equivalently reducing the standard deviation by a factor of $\sqrt{n}$. For $n = 100$ this means averaging $100$ iid samples reduces the standard deviation by a factor of $10$. This is the basic phenomenon responsible for the sample mean becoming a more and more accurate estimate of the true mean $\mu$, with higher and higher probability, as $n \to \infty$, in a way which is independent of the size of the population from which we are sampling; the population could even be infinite.
The simplest theorem that quantifies this is Chebyshev's inequality, which when applied here tells us that
$$\mathbb{P}(|S_n - \mu| \ge k \sigma) \le \frac{1}{nk^2}$$
which in words says that the probability that $S_n$ is more than $k$ times the standard deviation $\sigma$ of $X$ away from its mean is at most $\frac{1}{nk^2}$; for example, when $n = 100, k = 2$ this says that the probability that $S_{100}$ is more than $2 \sigma$ away from its mean is at most $\frac{1}{400} = 0.25 \%$.
The central limit theorem suggests, though, that with stronger hypotheses on $X$ it ought to be possible to write down much stronger bounds on these tail probabilities. For example, if $X \sim N(\mu, \sigma)$ is normal you can explicitly compute that the sample mean $S_n \sim N(\mu, \frac{\sigma}{\sqrt{n}})$ is also normal, but with standard deviation reduced by a factor of $\sqrt{n}$ as in the above calculation, which implies a much stronger version of the above bound which decays faster than exponentially in $k$, rather than only quadratically. There are various inequalities which imply bounds like this depending on what you want to assume about $X$, for example the Chernoff bound, Hoeffding's inequality, or the Berry-Esseen theorem, which is a quantitative version of the central limit theorem that gives bounds for finite $n$.

Some additional comments on the way these bounds depend only on the size of the sample and not on the size of the population. I think many people find this surprising at first; it may seem like this is too good to be true, like in some way you're learning too much information without putting in an equivalent level of effort. This is sort of paradoxical although I don't know if this paradox has a formal name.
The resolution is that the assumption that the survey actually randomly samples the population is extremely strong; the work it takes to actually do this is what's responsible for the strength of the results. This assumption essentially never holds in practice, and unfortunately all of these calculations depend crucially on this assumption. Actual surveys likely don't even get close to random sampling; you can always find sources of sample bias if you poke at them. So in that sense the relevance of these calculations to understanding actual surveys is questionable. This is really a discussion of the "ideal" survey, the one which hypothetically actually randomly samples the population.
