Central limit theorem does not work in this case?

Question

Say we have a dataset that contains 1 million values and Every single value is equal to $ 5$.

Say we take take a large number of samples from this set, and work out the mean of the sample each time. According to the central limit theorem, as the sample size approaches $\infty$, the distribution of sample means should tend to a normal distribution.

However, thinking about this logically, the sample mean is going to be $5$ every single time, and therefore the distribution of sample means will not be normally distributed.

How does this work?

Answer 1

You have a dataset whose "random" variables are drawn from the pdf $p(x) = \delta(x-5)$ (I'm infering this, you've provided 1 million samples with such a rigid structure and asked a question which implies your data is 'certain' to be 5).

The delta "function" is the distribution you get if you take a Gaussian distribution and send its variance to 0 (in the appropriately careful mathematical sense).

One nice, somewhat physical, way to see this is to note that the Gaussian distribution with variance $\sigma^{2} = t$ solves the Heat Equation $\partial_{t}p(x,t) = \partial_{x}^{2}p(x,t)$ with initial condition $p(x,t) = \delta(x)$. From this point of view a standard Gaussian is a diffused 'melted' version of the delta distribution.