One of the ways to state the (strong) Law of Large Numbers is this:

Given a probability space $(\Omega, \mathcal{A}, P)$,

$$\lim_{n \to \infty} \left( \frac{1}{n} \sum_{i=1}^{n} X_i \right) \overset{a.s.}{\longrightarrow} \mu$$

Where $(X_i)_{1 \leq i < n}$ are independent, identically distributed random variables with mean $\mu$.

But the random variables are (measurable) functions $X : \Omega \mapsto \mathbb{R}$. So, my question is: in this (and similar) presentations of the LLN, what do the $X_i$ represent in terms of the events $\omega \in \Omega$? Are we assuming the same event for all of them (the $X_i$ are really $X_i(\omega)$ for a fixed $\omega$)? Or are we summing over all the events for the same random variable (the $X_i$ are really $X(\omega_i)$ for the same $X$ and different $\omega_i \in \Omega$)? Or something else?

I understand that this is a notation confusion on my part, but I want to know what exactly is being stated here.

For example, I know that whenever we write $P(X_i = x_i)$, we really mean $P(\{\omega\})$, where $\{\omega \in \Omega : X_i(\omega) = x_i\}$, since $P: \mathcal{A} \mapsto [0,1]$. I would like a similar break down of the $X_i$ in the LLN (or Markov Chain analysis etc.)

  • 5
    $\begingroup$ Yes, the $X_i$ are measurable functions. So the conclusion of the LLN means that $\frac1n\sum_{i=1}^n X_i(\omega)\to\mu$ for almost every $\omega\in\Omega$. $\endgroup$ – David C. Ullrich May 25 '18 at 11:39
  • 1
    $\begingroup$ @DavidC.Ullrich: That looks like a full answer. $\endgroup$ – Henning Makholm May 25 '18 at 11:39
  • $\begingroup$ @HenningMakholm I guess. I'd prefer that if someone is browsing my Answers they see things with more content. If I made that comment an Answer it would be expanded: (i) I'd want to mention the Rademacher functions as a concrete example, (ii) I'd want to explicitly address the OP's confusion instead of just saying how things are: A priori the $"X_i(\omega_i)$" doesn't really make much sense, but given that the $X_i$ are iid it's possible to "realize" things in such a way that $\Omega=\prod\Omega_j$ and in fact $X_i(\omega)=\omega_i$. $\endgroup$ – David C. Ullrich May 25 '18 at 12:12
  • $\begingroup$ @DavidC.Ullrich Your comment did clarify things to me. I'm just a bit uncomfortable with the "almost every $\omega$" part... $\endgroup$ – usernameiwantedwasalreadytaken May 25 '18 at 12:20
  • $\begingroup$ @usernameiwantedwasalreadytaken: "almost every $\omega$" is standard jargon for "a set of $\omega$s whose probability measure is $1$". $\endgroup$ – Henning Makholm May 25 '18 at 12:21

The $X_i$s are measurable functions, but it is common in probability calculations not to write the argument to those functions explicitly because it's always $\omega$.

(We can explain that either as a notational convention, or we can say that we're extending every mathematical operator to work pointwise on the space of measurable functions from $\Omega$).

The "a.s." on the arrow means "almost surely", standard jargon for "in an event of measure $1$".

With these conventions unfolded, $$\lim_{n \to \infty} \left( \frac{1}{n} \sum_{i=1}^{n} X_i \right) \overset{a.s.}{\longrightarrow} \mu$$ means precisely $$ P\left(\biggl\{\omega\in\Omega \Biggm| \lim_{n \to \infty} \biggl( \frac{1}{n} \sum_{i=1}^{n} X_i(\omega) \biggr) = \mu \biggr\}\right) = 1 $$

  • $\begingroup$ Ok. So suppose we perform an experiment and a measurement encoded in the random variable $X$ and that we repeat this experiment $n$ times each time in the same conditions and each time independently of each other. Each time we may observe a different outcome of the experiment, $\omega$. When @DavidC.Ullrich first commented, I thought we could just sum $X(\omega)$ for all the experiments (which is how a Monte Carlo simulation is made, if I'm not mistaken). But the way you put this, seems that we will have a set of $\omega$'s and, for each of them, the emprical average will tend to $\mu$. $\endgroup$ – usernameiwantedwasalreadytaken May 25 '18 at 12:39
  • 1
    $\begingroup$ @usernameiwantedwasalreadytaken: In the usual formalism there is a single $\omega$ that encodes the result of all the (countably many) experiments in your super-experiment. Each of the $X_i$s is a _different_ function from $\Omega$ to (say) $\mathbb R$, such that $X_i(\omega)$ can depend on $i$ even when we hold $\omega$ fixed. That's why we're not saying that they're the same random variable, but merely identically distributed random variables. $\endgroup$ – Henning Makholm May 25 '18 at 12:49
  • $\begingroup$ Before we start the sequence of experiments the law promises us that no matter how the sequence turns out -- that is, no matter which single $\omega$ the gods of chance pick -- we can be pretty damn sure that the sequence of sample averages will converge to $\mu$, where "pretty damn sure" means "with probability $1$". $\endgroup$ – Henning Makholm May 25 '18 at 12:54
  • $\begingroup$ Ok. It makes sense now. Thank you (by the way, I didn't mean to sound rude before, I'm really thankful for your explanations). $\endgroup$ – usernameiwantedwasalreadytaken May 25 '18 at 12:55
  • $\begingroup$ @usernameiwantedwasalreadytaken: I don't think you sounded rude anywhere. $\endgroup$ – Henning Makholm May 25 '18 at 12:57

Each $X_i$ is a map e.g. $X_i : \Omega \to R$. Then you could consider $\prod_i X_i$ a map from $\prod_i\Omega_i$ to $\prod_iR_i$ which just means that for one event $\prod_i w_i$ you have a sequence of real numbers the convergance in probability means that $$ (\prod_i P_i)(\{\prod_iw_i : \frac{1}{n}\sum_{i=1}^{\infty}X_i(w_i) \to \mu\}) = 1 $$ You need to be a bit careful in how to define these, see references below.

To read more see http://planetmath.org/infiniteproductmeasure

Infinite product of measurable spaces

Now for the Markov chain example, which I missed, sorry!, The sequence is dependent and for those you need to model dependance, one example to do this is to consider models of the kind $$ X_{i+1} = F(X_{i}) + Z_{i}, X_0 = 0. $$ with $Z_i$ i.i.d. and hence defined in the product sense above and then from this i.i.d. and a matematical derivation apply the measure. One could say that a common theme in processes modeling is to from a set of independent processes construct a model with dependency that accurately fit the real world example. e.g. see ARMA Models and Kalman filters.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.