Definition of Sampling I am no expert in statistics, I know statistics at engineering level.
I've been reading through measure theory and probability theory lately (still not an expert but definitely more solid than my statistics background at the moment).
Suppose you have a measure space $(\Omega,\mathcal{F},\mu)$ where $\mu$ is a probability measure. Suppose $X$ is a random variable. What I am looking for is a rigorous (if there's any) definition of "sampling" of that random variable. Or maybe what I am looking is more how to get "samples" from a probability distribution (again a rigorous definition).
I know a standard "algorithm" to perform sampling, however I am more looking for a mathematical definition of sampling.
I am either looking for an answer or a some rigorous definition somewhere.
Just to give an analogue in signal processing if you have a signal $s : \mathcal{T} \to \mathbb{R}$ (time set to real values) we can define the sampling at rate say $T = \frac{1}{f}$ as
$\left\{ s(nT) \right\}_{n \in \mathbb{Z}}$ and this is a well understood mathematical definition to me which I can rephrase as composition of $s$ with the map $I_T : \mathbb{N} \to \mathcal{T}$ that given $n$ returns $nT$ hence
$$
s(nT) = \left(s \circ I_T\right)(n)
$$
I do struggle however with sampling in probability terms.
Can you help?
 A: Despite dealing with randomness, probability theory doesn’t specify a method to identify which outcome will happen for a particular realization of a random variable. The notion of randomness is inherent in the probability measure itself — each outcome gets a piece of the overall probability.
That being said, we could say that sampling is a particular choice function over $\Omega$
$$f: \Omega \to \Omega$$
With this construct we can say that a sample of size $n$ from $\Omega$ is simply applying $f$ to a collection of $n$ copies of $\Omega$.
Of course, most choices for $f$ will lead to “biased” samples. We need some additional constraints on $f$ to ensure it generates representative draws from the sample space.
The closest thing I can think of is the Glivenko-Cantelli theorem  for the almost sure convergence of empirical measures.
With this theorem we can say that a sample of size $n$ is the application of a choice function $q$ to the collection of $n$ copies of $\Omega$ such that  the Glivenko-Cantelli theorem holds for any empirical measure defined on subsets of $\Omega$ as the number of copies of $\Omega$ approaches infinity.
$$\lim_{n\to\infty}\sup_{s\in\mathcal{F}}|\mu_n(s)  - P(s)| = 0 $$
$$\text{where} \; \mu_n:= \mu(s; q(\mathcal{C}_n)),\; \mathcal{C}_n:= \{\Omega_i\}_1^n,\;\; \Omega_i = \Omega_j \;\;\forall i\neq j$$
