In the book I'm currently reading, this law is stated as:

The Weak Law of Large Numbers provides proof of the notion that if $n$ independent and identically distributed random variables, $X_1,X_2,...,X_n$, from a distribution with finite variance are observed, then the sample mean, $\bar{X}$, should be very close to $\mu$ provided $n$ is large.

The problem is, the book hasn't formally defined $\bar{X}$, so I'm unclear as the the relationship between $X$ and the $X_i's$. From the context I assuming that:

$$\bar{X} = \frac{X_1+X_2+\cdots+X_n}{n}$$

But this makes no sense to me since the $X_i's$ may not necessarily all even be the same unit, so this would be like adding apples and oranges. In other words, I can say, in an experiment of three coin tosses, let $X_1$ be the number of tails that appear and $X_2$ be the number of heads that appear, thus the units for $X_1$ and $X_2$ are tails and heads, respectively.

Understanding the relationship between the $X_i's$ and $X$ is important to me to make sense of the expression:

$$\lim_{n \rightarrow \infty} \mathbb P \left( \left| \frac{X_1+X_2+\cdots+X_n}{n}-\mu \right| \ge \epsilon \right) = 0$$

which, upon replacing $\mu$ with its definition, I obtain:

$$\lim_{n \rightarrow \infty} \mathbb P \left( \left| \frac{X_1+X_2+\cdots+X_n}{n}-E[X] \right| \ge \epsilon \right) = 0$$

Perhaps this is totally wrong but my interpretation of this law is essentially if $\mu_i=E[X_i]$, then, as $n \rightarrow \infty$, $\mu_1=\mu_2=\cdots=\mu_n=\mu$, which I feel is intuitively obvious as all the $X$'s are identically distributed (again, ignoring units).

So my questions boil down to:

  1. What is the definition of $\bar{X}$?
  2. What is the conceptual meaning of $\frac{X_1+X_2+\cdots+X_n}{n}$?
  3. How do I reconcile the unit differences of the expression in question 2?
  4. What is the relationship between the $X_i$'s and $X$?
  • 1
    $\begingroup$ I don't know is it a typo or your misunderstanding, but WLLN is $\lim_{n \rightarrow \infty} \mathbb P \left( \left| \frac{X_1+X_2+...X_n}{n}-\mu \right| \ge \epsilon \right) = 0$ $\endgroup$
    – kludg
    Jul 11 '20 at 17:53
  • $\begingroup$ $\overline X_n$ would be a better notation for $(X_1+\cdots+X_n)/n$ $\endgroup$ Jul 11 '20 at 17:57
  • $\begingroup$ In your coin example, $X_1$ and $X_2$ have the same distribution, but they're not independent, are they? If I understand you correctly, they are related by $X_1+X_2 = 3$, so (for example) $P(X_1 = 1, X_2 = 1) = 0 \not= 9/64 = P(X_1 = 1)P(X_2 = 1)$. $\endgroup$
    – Brian Tung
    Jul 11 '20 at 18:21

Your definition of $\bar{X}$ is correct. I think you've just chosen a poor example to try to apply it to.

In your first example, if $X_1$ is number of heads and $X_2$ is number of tails, I would argue that both values have units of "coins", so it does make sense to add them. $X_1 + X_2$ is the total number of coins which are showing either heads or tails (which is always 3). But they are not independent so the weak law of large number will not have anything to say about this sum.

For a better example, suppose you roll $n$ dice, and $X_i$ is the number of pips showing on the $i$th die. These all have the same units (pips) and $X_1 + \dots + X_n$ is the total number of pips showing. On different runs of the experiment, it could be anywhere between $n$ and $6n$. Then $\bar{X} = \frac{X_1 + \dots + X_n}{n}$ is the (empirical) average number of pips per die. On different runs of the experiment, it could be anywhere between 1 and $n$.

The weak law of large numbers says that when $n$ is very large, this observed value is unlikely to be very far from its theoretical mean of $3.5$.

  • $\begingroup$ I see, I understand your point. I guess then I simply don't truly get what $\bar{X}$ is then. My understanding is that the $X_i$'s are functions, not specific values, so I don't know exactly how to compute $\bar{X}$. $\endgroup$
    – Isaac
    Jul 11 '20 at 21:28
  • $\begingroup$ @Isaac: Indeed, the $X_i$ are functions, whose inputs are elements of the sample space. When you add two functions, you get another function. Think of my example with two dice. The possible outcomes could be written as $(1,1), (1,2), \dots, (6,6)$. So if outcome $(3,4)$ happens, then $X_1$ is 3 and $X_2$ is $4$, which is to say $X_1(3,4) = 3$ and $X_2(3,4) = 4$. So that means $$\bar{X}(3,4) = \frac{X_1(3,4) + X_2(3,4)}{2} = \frac{3+4}{2} = 3.5.$$ You could similarly write down the output values of the function $\bar{X}$ for every other outcome, and see exactly what function it is. $\endgroup$ Jul 11 '20 at 21:38
  • $\begingroup$ Okay, that makes sense. So basically $\bar{X}$ is the empirical mean after $n$ experiments whereas $\mu$ is the theoretical mean? If so, you're counting each rolled dice as an "experiment" and not the roll of both dice as a single experiment? $\endgroup$
    – Isaac
    Jul 12 '20 at 3:23
  • $\begingroup$ No, I do consider both rolls to constitute a single experiment. It only makes sense to add random variables coming from the same experiment, otherwise you are trying to add functions that have different domains. You can consider the WLLN as speaking about a sequence of experiments with more and more dice, or about a single experiment that has infinitely many dice but you consider the empirical average of finitely many at a time. $\endgroup$ Jul 12 '20 at 3:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.