# What is the difference between "expectation", "variance" for statistics versus probability textbooks?

It seems that there are two ideas of expectation, variance, etc. going on in our world.

In any probability textbook:

I have a random variable $$X$$, which is a function from the sample space to the real line. Ok, now I define the expectation operator, which is a function that maps this random variable to a real number, and this function looks like, $$\mathbb{E}[X] = \sum\limits_{i = 1}^n x_i p(x_i)$$ where $$p$$ is the probability mass function, $$p: x_i \mapsto [0,1], \sum_{i = 1}^n p(x_i) = 1$$ and $$x_i \in \text{range}(X)$$. The variance is, $$\mathbb{E}[(X - \mathbb{E}[X])^2]$$

The definition is similar for a continuous RV.

However, in statistics, data science, finance, bioinformatics (and I guess everyday language when talking to your mother)

I have a multi-set of data $$D = \{x_i\}_{i = 1}^n$$ (weight of onions, height of school children). The mean of this dataset is

$$\dfrac{1}{n}\sum\limits_{i= 1}^n x_i$$

The variance of this dataset (according to "science buddy" and "mathisfun dot com" and government of Canada) is,

$$\dfrac{1}{n}\sum\limits_{i= 1}^n(x_i - \sum\limits_{j= 1}^n \dfrac{1}{n} x_j)^2$$

I mean, I can already see what's going on here (one is assuming uniform distribution), however, I want an authoritative explanation on the following:

1. Is the distinction real? Meaning, is there a universe where expectation/mean/variance... are defined for functions/random variables and another universe where expectation/mean/variance... are defined for raw data? Or are they essentially the same thing (with hidden/implicit assumption)

2. Why is it no probabilistic assumption is made when talking about mean or variance when it comes to dealing with data in statistics or data science (or other areas of real life)?

3. Is there some consistent language for distinguishing these two seemingly different mean and variance terminologies? For example, if my cashier asks me about the "mean weight" of two items, do I ask him/her for the probabilistic distribution of the random variable whose realization are the weights of these two items (def 1), or do I just add up the value and divide (def 2)? How do I know which mean the person is talking about?/

• There's no probabilistic assumption about the sample mean or sample variance because they're just numbers computed from the sample data. If you measure the heights of $100$ people, you have $100$ numbers and you can do whatever computations you like with them. Commented Dec 23, 2020 at 5:07
• Can you explain what you what you mean that 'uniform distribution' was assumed? Isnt p(x_i)=lim n->infty #occurences of x_i/n ? So an approximation of p(x_i) is #occurences of x_i /n. So 1/n sum x_i = 1/n sum_{x_i} x_i #occurences of x_i = sum x_i p(x_i). Of course p(x_i) is only an approximation of p(x_i) in statistics. I do not see how uniformity was used here Commented Dec 23, 2020 at 16:25
• "Why is it no probabilistic assumption is made when talking about mean or variance when it comes to dealing with data in statistics or data science " Can you be more clear as to what assumption you think is made elsewhere, but isn't made in data science? Commented Dec 24, 2020 at 20:46
• @lalala If you take the average off $n$ values, that is equal to the expected value of a random variable that has a uniform distribution across those alues. Commented Dec 24, 2020 at 20:47

You ask a very insightful question that I wish were emphasized more often.

EDIT: It appears you are seeking reputable sources to justify the above. Sources and relevant quotes have been provided.

Here's how I would explain this:

• In probability, the emphasis is on population models. You have assumptions that are built-in for random variables, and can do things like saying that "in this population following such distribution, the probability of this value is given by the probability mass function."
• In statistics, the emphasis is on sampling models. With most real-world data, you do not have access to the data-generating process governed by the population model. Probability provides tools to make guesses on what the data-generating process might be. But there is always some uncertainty behind it. We therefore attempt to estimate characteristics about the population given data.

From Wackerly et al.'s Mathematical Statistics with Applications, 7th edition, chapter 1.6:

The objective of statistics is to make an inference about a population based on information contained in a sample taken from that population...

A necessary prelude to making inferences about a population is the ability to describe a set of numbers...

The mechanism for making inferences is provided by the theory of probability. The probabilist reasons from a known population to the outcome of a single experiment, the sample. In contrast, the statistician utilizes the theory of probability to calculate the probability of an observed sample and to infer this from the characteristics of an unknown population. Thus, probability is the foundation of the theory of statistics.

From Shao's Mathematical Statistics, 2nd edition, section 2.1.1:

In statistical inference... the data set is viewed as a realization or observation of a random element defined on a probability space $$(\Omega, \mathcal{F}, P)$$ related to the random experiment. The probability measure $$P$$ is called the population. The data set or random element that produces the data is called a sample from $$P$$... In a statistical problem, the population $$P$$ is at least partially unknown and we would like to deduce some properties of $$P$$ based on the available sample.

So, the probability formulas of the mean and variance assume you have sufficient information about the population to calculate them.

The statistics formulas for the mean and variance are attempts to estimate the population mean and variance, given a sample of data. You could estimate the mean and variance in any number of ways, but the formulas you've provided are some standard ways of estimating the population mean and variance.

Now, one logical question is: why do we choose those formulas to estimate the population mean and variance?

For the mean formula you have there, one can observe that if you assume that your $$n$$ observations can be represented as observed values of independent and identically distributed random variables $$X_1, \dots, X_n$$ with mean $$\mu$$, $$\mathbb{E}\left[\dfrac{1}{n}\sum_{i=1}^{n}X_i \right] = \mu$$ which is the population mean. We say then that $$\dfrac{1}{n}\sum_{i=1}^{n}X_i$$ is an "unbiased estimator" of the population mean.

From Wackerly et al.'s Mathematical Statistics with Applications, 7th edition, chapter 7.1:

For example, suppose we want to estimate a population mean $$\mu$$. If we obtain a random sample of $$n$$ observations $$y_1, y_2, \dots, y_n$$, it seems reasonable to estimate $$\mu$$ with the sample mean $$\bar{y} = \dfrac{1}{n}\sum_{i=1}^{n}y_i$$

The goodness of this estimate depends on the behavior of the random variables $$Y_1, Y_2, \dots, Y_n$$ and the effect this has on $$\bar{Y} = (1/n)\sum_{i=1}^{n}Y_i$$.

Note. In statistics, it is customary to use lowercase $$x_i$$ to represent observed values of random variables; we then call $$\frac{1}{n}\sum_{i=1}^{n}x_i$$ an "estimate" of the population mean (notice the difference between "estimator" and "estimate").

For the variance estimator, it is customary to use $$n-1$$ in the denominator, because if we assume the random variables have finite variance $$\sigma^2$$, it can be shown that $$\mathbb{E}\left[\dfrac{1}{n-1}\sum_{i=1}^{n}\left(X_i - \dfrac{1}{n}\sum_{j=1}^{n}X_j \right)^2 \right] = \sigma^2\text{.}$$ Thus $$\dfrac{1}{n-1}\sum_{i=1}^{n}\left(X_i - \dfrac{1}{n}\sum_{j=1}^{n}X_j \right)^2$$ is an unbiased estimator of $$\sigma^2$$, the population variance.

It is also worth noting that the formula you have there has expected value $$\dfrac{n-1}{n}\sigma^2$$ and $$\dfrac{n-1}{n} < 1$$ so on average, it will tend to underestimate the population variance.

From Wackerly et al.'s Mathematical Statistics with Applications, 7th edition, chapter 7.2:

For example, suppose that we wish to make an inference about the population variance $$\sigma^2$$ based on a random sample $$Y_1, Y_2, \dots, Y_n$$ from a normal population... a good estimator of $$\sigma^2$$ is the sample variance $$S^2 = \dfrac{1}{n-1}\sum_{i=1}^{n}(Y_i - \bar{Y})^2\text{.}$$

The estimators for the mean and variance above are examples of point estimators. From Casella and Berger's Statistical Inference, Chapter 7.1:

The rationale behind point estimation is quite simple. When sampling is from a population described by a pdf or pmf $$f(x \mid \theta)$$, knowledge of $$\theta$$ yields knowledge of the entire population. Hence, it is natural to seek a method of finding a good estimator of the point $$\theta$$, that is, a good point estimator. It is also the case that the parameter $$\theta$$ has a meaningful physical interpretation (as in the case of a population) so there is direct interest in obtaining a good point estimate of $$\theta$$. It may also be the case that some function of $$\theta$$, say $$\tau(\theta)$$ is of interest.

There is, of course, a lot more that I'm ignoring for now (and one could write an entire textbook, honestly, on this topic), but I hope this clarifies things.

Note. I know that many textbooks use the terms "sample mean" and "sample variance" to describe the estimators above. While "sample mean" tends to be very standard terminology, I disagree with the use of "sample variance" to describe an estimator of the variance; some use $$n - 1$$ in the denominator, and some use $$n$$ in the denominator. Also, as I mentioned above, there are a multitude of ways that one could estimate the mean and variance; I personally think the use of the word "sample" used to describe such estimators makes it seem like other estimators don't exist, and is thus misleading in that way.

## In Common Parlance

This answer is informed primarily by my practical experience in statistics and data analytics, having worked in the fields for about 6 years as a professional. (As an aside, I find one serious deficiency with statistics and data analysis books is providing mathematical theory and how to approach problems in practice.)

Is there some consistent language for distinguishing these two seemingly different mean and variance terminologies? For example, if my cashier asks me about the "mean weight" of two items, do I ask him/her for the probabilistic distribution of the random variable whose realization are the weights of these two items (def 1), or do I just add up the value and divide (def 2)? How do I know which mean the person is talking about?

In most cases, you want to just stick with the statistical definitions. Most people do not think of statistics as attempting to estimate quantities relevant to a population, and thus are not thinking "I am trying to estimate a population quantity using an estimate driven by data." In such situations, people are just looking for summaries of the data they've provided you, known as descriptive statistics.

The whole idea of estimating quantities relevant to a population using a sample is known as inferential statistics. While (from my perspective) most of statistics tends to focus on statistical inference, in practice, most people - especially if they've not had substantial statistical training - do not approach statistics with this mindset. Most people whom I've worked with think "statistics" is just descriptive statistics.

In descriptive data analysis, a few summary measures may be calculated, for example, the sample mean... and the sample variance... However, what is the relationship between $$\bar{x}$$ and $$\theta$$ [a population quantity]? Are they close (if not equal) in some sense? The sample variance $$s^2$$ is clearly an average of squared deviations of $$x_i$$'s from their mean. But, what kind of information does $$s^2$$ provide?... These questions cannot be answered in descriptive data analysis.

## Other remarks about the sample mean and sample variance formulas

Let $$\bar{X}_n$$ and $$S^2_n$$ denote the sample mean and sample variance formulas provided earlier. The following are properties of these estimators:

• They are unbiased for $$\mu$$ and $$\sigma^2$$, as explained earlier. This is a relatively simple probability exercise.
• They are consistent for $$\mu$$ and $$\sigma^2$$. Since you know measure theory, assume all random variables are defined over a probability space $$(\Omega, \mathcal{F}, P)$$. It follows that $$\bar{X}_n \overset{P}{\to} \mu$$ and and $$S^2_n \overset{P}{\to} \sigma^2$$, where $$\overset{P}{\to}$$ denotes convergence in probability, also known as convergence with respect to the measure $$P$$. See https://math.stackexchange.com/a/1655827/81560 for the sample variance (observe that the estimator with the $$n$$ in the denominator is used here; simply multiply by $$\dfrac{n-1}{n}$$ and apply a result by Slutsky) and Proving a sample mean converges in probability to the true mean for the sample mean. As a stronger result, convergence is almost sure with respect to $$P$$ in both cases (Sample variance converge almost surely).
• If one assumes $$X_1, \dots, X_n$$ are independent and identically distributed based on a normal distribution with mean $$\mu$$ and variance $$\sigma^2$$, one has that $$\dfrac{\sqrt{n}(\bar{X}_n - \mu)}{\sqrt{S_n^2}}$$ follows a $$t$$-distribution with $$n-1$$ degrees of freedom, which converges in distribution to a normally-distributed random variable with mean $$0$$ and variance $$1$$. This is a modification of the central limit theorem.
• If one assumes $$X_1, \dots, X_n$$ are independent and identically distributed based on a normal distribution with mean $$\mu$$ and variance $$\sigma^2$$, $$\bar{X}_n$$ and $$S^2_n$$ are uniformly minimum-variance unbiased estimators (UMVUEs) for $$\mu$$ and $$\sigma^2$$ respectively. It also follows that $$\bar{X}_n$$ and $$S^2_n$$ are independent, through - as mentioned by Michael Hardy - showing that $$\text{Cov}(\bar{X}_n, X_i - \bar{X}_n) = 0$$ for each $$i = 1, \dots, n$$, or as one can learn from more advanced statistical inference courses, an application of Basu's Theorem (see, e.g., Casella and Berger's Statistical Inference).
• I believe the expressions for mean and variance estimators should be accompanied with a word or two on the central limit theorem. Commented Dec 23, 2020 at 5:52
• You wrote: "If one assumes $X_1,\ldots,X_n$ are normally distributed, $\bar X_n$ and $S_n^2$ are" UMVUEs. (This after assuming they are i.i.d.) I would add that under those assumptions $\bar X_n$ and $S_n^2$ are independent, and that is used in deriving the t-distribution. One of the quickest ways to show that involves showing that $\operatorname{cov} \left(\,\overline X, X_i-\overline X\,\right)=0.$ When $U,V$ are both linear combinations of i.i.d. normals, then their covariance is $0$ only if they are independent. Commented Dec 26, 2020 at 21:47
• @MichaelHardy Thanks for correcting me; I've put in the edits. Commented Dec 26, 2020 at 22:08
• @Clarinetist : I see you say it's "through an application of Basu's theorem," but nothing so advanced as Basu's theorem is needed, since the method described in my comment above is enough. It seems misleading to tacitly suggest that you can't learn to prove that proposition until you learn Basu's theorem. Commented Dec 26, 2020 at 22:11
• @MichaelHardy Thanks, I've corrected that as well. Commented Dec 26, 2020 at 22:13

The first definitions you gave are correct and standard, and statisticians and data scientists will agree with this. (These definitions are given in statistics textbooks.) The second set of quantities you described are called the "sample mean" and the "sample variance", not mean and variance.

Given a random sample from a random variable $$X$$, the sample mean and sample variance are natural ways to estimate the expected value and variance of $$X$$.

• Thanks! So the distinction is one is "sample" mean/variance the other is mean/variance as it is. And also those formulas are estimate of the value of mean and variance. But isn't it true when people are working with them in practice, say using some data science package such as Scikit-Learn, the underlying probabilistic assumption (such as distribution of data) is dropped as if nothing ever happened? I have never seen anyone writing a documentation mentioning the probabilistic definition. I think this causes confusion for young practictioners, don't you think? Commented Dec 23, 2020 at 5:39
• Imagine in the shoes of a young data scientist who just took a course on Lebesgue integration or measure theory. You spent 4 month talking about these abstract definitions, and then integrating these really hard integrals just to get the mean, using various tricks just to make these integrals tractable. You get your first job and your boss tells you just to add stuff up and divide. Isn't this confusing? Commented Dec 23, 2020 at 5:41
• @Norman My response to your comments here is that probability is not a statistics class, and trying to act as if that is the case would be lying. Additionally, most managers have not taken courses nor understand the difference between probability and statistics, and are focused solely on descriptive statistics to understand empirically the behavior of their data. They calculate a sample average because that's what they habitually have done, and they think it is an adequate summary of the data. Commented Dec 23, 2020 at 13:25
• @Norman I could go on a very long rant about how poorly taught statistics is in many schools and how very little guidance is given on how to approach statistical problems in practice, but I will avoid that for now. Commented Dec 23, 2020 at 13:27
• @Norman I think it can be a source of confusion that people often say mean or variance when they really mean "sample mean" or "sample variance". One of the challenges of coming into the engineering world from a pure math background is that often people speak much less precisely than one is used to. Commented Dec 23, 2020 at 15:23

Other answers — particularly Clarinetist’s — give excellent rundowns of the most important side of the answer. Given a random variable, we can sample it, and use the sample mean (defined in the statistical sense) to estimate the actual mean of the random variable (defined in the sense of probability theory), and similarly for variance, etc.

But the connection in the other direction doesn’t seem to have been mentioned yet. This is not as important, but it’s much more straightforward, and worth pointing out. Given a sample, i.e. a finite multiset of values $$\{x_i\}_{i \in I}$$, we can “consider this as a distribution”, i.e. take a random variable $$X$$, with value $$x_i$$ for $$i$$ distributed uniformly over $$I$$. Then the mean, variance, etc. of $$X$$ (in the sense of probability theory) will be precisely the mean, variance, etc. of the original multiset (defined in the statistical sense).

The confusion actually comes from notation, where symbols mean different things in two formulas.

First, let's take a look at the "probabilistic" definition: $$\mathbb{E}[X] = \frac{1}{n}\sum_{i=1}^n x_i p(x_i)$$ Here the random variable $$X$$ takes $$n$$ distinct values $$x_1,\ldots, x_n$$, each with a probability mass function $$p(x_i)$$.

In the "statistical" definition we have an estimate of the expected value, based on the observed values of the random variable $$z_1, \ldots, z_N$$: $$\hat{\mathbb{E}}[X] = \frac{1}{N}\sum_{j=1}^N z_j$$ Notice that I renamed the variables compared to your original formula, so as to avoid the confusion. Here $$N$$ is the number of observations, and $$z_j$$s are the actual observations. For example, if $$X$$ is a random variable representing rolls of a loaded die, then $$n = 6$$ and $${x_i} = \{1, 2, 3, 4, 5, 6\}$$; whereas $$N$$ can be arbitrarily large, and $$z_j$$s will just be a long sequence of random draws from the set of $${x_i}$$s.

Now, you can rewrite the "statistical" formula by collecting different values of $$z_j$$s into the groups according to which $$x_i$$ they correspond to (for example, first collect all 1s, then all 2s, etc): $$\hat{\mathbb{E}}[X] = \frac{1}{N}\big(x_1\cdot|\{z_j=x_1\}| + \cdots+x_n\cdot|\{z_j=x_n\}|\big) \\ =\frac{1}{N}\sum_{i=1}^n x_i \sum_{j=1}^N\mathbb{1}[z_j=x_i] \\ =\sum_{i=1}^n x_i \hat{p}(x_i)$$ where $$\hat{p}(x_i) = \frac{1}{N}\sum_{j=1}^N \mathbb{1}[z_j=x_i]$$ is the estimate of the probability mass function: the number of times a value $$x_i$$ was encountered in the sample divided by the sample size, i.e. the observed frequency of the value $$x_i$$.

Now you can see that the "probabilistic" and "statistical" definitions are actually the same, with the only difference that we replace the theoretical mass distribution function $$p(x)$$ (which may not be known) with the empirical (observed) mass distribution function $$\hat{p}(x)$$.

The general expression for arithmetic mean is $$\frac{\sum\limits_{i= 1}^n w_i x_i}{\sum\limits_{i= 1}^n w_i}$$, or even more generally, $$\frac{\int w(t) f(t)dt}{\int f(t)dt}$$ (there are then ways to recover the discrete case from that).

If you set all $$w_i$$ to $$1$$, or really to anything as long as it's constant, you get $$\dfrac{1}{n}\sum\limits_{i= 1}^n x_i$$. This is referred to as an "unweighted" average, although technically it's still weighted, it's just that you're multiplying everything by $$1$$, so you don't notice it. If you set $$p_i = \frac{w_i}{ \sum\limits_{k= 1}^n w_k}$$, and interpret $$p_i$$ as the probability of the $$i$$th event, then you get the average weighted by probability, which is also known as expected value.

You have to be careful about "unweighted" averages, as they often actually are weighted, but by weights that you didn't want. For instance, suppose you want the average income over the US, and you have the average income for each state individually. You could just add all of those averages together, and then divide by $$50$$. People will often call this the "unweighted" or "simple" average, but you're actually weighting people by the reciprocal of their states population; the fewer people there are in a state, the more each person's income affects that state's average, and so the more they affect the total "unweighted average". As a result, to get the actual overall average income from the individual states' averages, you have to multiply each state's average by its population to get its total income, add all of those together, and then divide by the total population.

A common weighting you'll see is frequency weighting. This is where you multiply each value by the number of times it appears. For instance, if you measure something once a month for a year, and the only values you get are $$0$$, $$1$$, and $$2$$, taking the simple average of those values gives you $$1$$. But that's probably not the real average. To get a more meaningful average, you should take each of these values, weight them by how many months they appear for, and then take the average.

One property of weighted averages is that multiplying all of the weights by a constant number doesn't change the final result (you'll just divide it out again when you divide by the total of the weights). So weighting by the frequencies is equivalent to weighting by the percentage of cases each value is. That is, if $$0$$ is the value for $$5$$ months, $$1$$ is the value for $$4$$ months, and $$2$$ is the value for $$3$$, the weighting of $$5,4,3$$ is equivalent to the weighting of $$\frac 5 {12},\frac 4{12},\frac 3{12}$$.

So if you have a probability distribution where one thing has a $$60$$% chance of happening, and something else has a $$40$$% chance of happening, the expected value is simply the frequency weighted average.

For example, if my cashier asks me about the "mean weight" of two items, do I ask him/her for the probabilistic distribution of the random variable whose realization are the weights of these two items (def 1), or do I just add up the value and divide (def 2)? How do I know which mean the person is talking about?

The expected value of a random variable is the expected value of that random variable. It is a property of the distribution. If you're asked for the expected value of a random variable, you find the expected value of that random variable. If you aren't asked for the expected value of a random variable, you don't go looking for a random variable to take the expected value of. How you know whether you take the expected value of random variable is if there is a random variable to take the expected value of. Even if values come from a distribution, the average of those values is the average of those values, not the average of the distribution they came from.

If people are talking rigorously, they will explicitly say they want the expected value. You many, however, see people asking for the "mean" or "average", when they really want the expected value, but you can recognize those cases by there being a random variable. For instance, if someone asks "What's the average payout of this slot machine?", the context suggests that you should take the expected value of the distribution of payouts, and not simply take the set of different possible payouts and take the simple average. There could be some ambiguity as to whether "the payout" refers to the random process that pays money out, in which case you should take the expected value of the distribution (population mean), or the actual money paid out, in which case you should take the average of all the actual payouts made by the machine (sample mean), but in the latter case, you still should take the frequency weighted average.

Statement: the "statistic" universe from your question is a partial case of "probabilistic" universe.

We need some notation. Suppose that we have a random variable $$\xi$$ in sense of probability theory and $$P(\xi = x_k) = p_k$$, $$0 \le k \le n$$.

For example, consider $$\xi$$ which is equal to the number of children in "abstract" random family. Put $$x_k = k$$. Then $$E \xi = \sum_{k=0}^n x_k p_k$$ where $$p_k = P(\xi = x_k)$$, and $$D\xi = E (\xi - E\xi)^2 = \sum_{k=0}^n p_k (x_k - E\xi )^2$$. Moreover, $$P(\xi = 69) > 0$$ (according to Guinness World Records, I am not sure, but let us suppose that it's a record nowadays) and as there was people with $$69$$ children then we may think that for example $$P(80) > 0$$ - it's natural, because $$80$$ children is possible, even although Guinness World Records says that still nobody had $$80$$ children.

In reality we don't have abstract random family, random mothers, random fathers and random children. We have finite number $$N$$ ( $$10^6 < N < 10^{100}$$) of families, let us number them, and there are $$y_1$$ children in the first family, there are $$y_2$$ children in family number $$2$$, ..., there are $$y_N$$ children in family $$N$$.

Analogy: $$\xi$$ corresponds to a fair dice itself and numbers $$y_1$$, $$y_2$$, ... corresponds to dice rolls and have form: $$5, 1, 6, 6, 3, ...$$, these are fixed numbers.

Now consider a random value $$\tau$$, which has uniform distribution on $$\{ 1, 2, \ldots, N\}$$. It means that $$P(\tau = k) = \frac{1}{N}$$ for all $$1 \le k \le N$$.

Numbers $$y_1, \ldots, y_N$$ are fixed - suppose they are written in some sociological table. Let us consider a random value $$y_{\tau}$$. It's not a number of children in abstract random family. It is a number of children in some real family, if a number $$\tau$$ of such family we chose randomly.

Let us find $$E y_{\tau}$$ and $$D y_{\tau}$$. With probability $$\frac{1}N$$ we have $$\tau = k$$ and hence $$y_{\tau} = y_k$$. Thus $$E y_{\tau}=\frac{1}N \sum_{k=1}^N y_k$$ and $$D y_{\tau} = E (y_{\tau} - Ey_{\tau})^2 = E (y_{\tau} - \frac{1}N \sum_{k=1}^N y_k )^2 = \frac{1}N \sum_{k=0}^N (y_k -\frac{1}N \sum_{k=1}^N y_k )^2.$$

Now the correspondence of "statistical universe" and "probabilistic universe" is obvious.

Notice that all $$y_i \le 69$$ and $$y_i = 69$$ is the maximum value, but the maximum value of $$\xi$$ is bigger (and not less than $$80$$).

So, we have shown that the "statistical universe" is a partial case of "probabilistic" universe.

Moreover, there are two moments, when there is randomness:

1. when we go from abstract random family, corresponding to $$\xi$$, to numbers $$y_1, \ldots, y_N$$ from sociological handbook.

As it was already mentioned, there is analogy: $$\xi$$ corresponds to a fair dice itself and numbers $$y_1$$, $$y_2$$, ... corresponds to dice rolls and have form: $$5, 1, 6, 6, 3, ...$$, these are fixed numbers.

1. [here we suppose that $$y_1, \ldots, y_N$$ are fixed numbers] when we go from the whole sociological handbook to a family with random number $$\tau$$ and see, how much children is there in this family. The number is $$y_{\tau}$$.

When from $$y_{\tau}$$ we go to $$E y_{\tau} = \frac{1}N \sum_{k=1}^N y_k$$, we get rid of the second randomness, but we still do not get rid of the first randomness. It's connected with the fact that numbers $$y_k$$ could be another: it could happen, that in some families could be more children, and in some families could be less children, if the circumstances were different. In this sence numbers $$y_i$$ are not fixed: they are r.v. sampled from distribution, corresponding to $$\xi$$. And in this sence $$\frac{1}N \sum_{k=1} y_k$$ is a random value, and we may write limit theorems such as L.L.N.: $$\frac{1}N \sum_{k=1}^N y_k \to E\xi, \text{ }N \to \infty$$ or CLT or LIL.

Addition: it was shown that if $$y_1, \ldots, y_N$$ are fixed and $$\tau$$ is random then "probabilistic" mean (expectation) of $$y_{\tau}$$ and "probabilistic" variance of $$y_{\tau}$$ are well known sample mean and sample variance.

Hope it will be useful. If you have any questions, you are welcome.