!!! Layman here so please avoid complex math and answers.

Random (usually pseudorandom) events are usually characterized along these lines:

  1. Each outcome in a trial experiment must be i.i.d.; i.e. it has no effect on subsequent outcomes, thus individual outcomes cannot be predicted using past data as there is no obvious causal link
  2. Large sequences of outcomes are predictable, because they exhibit a pattern of stabilizing relative frequencies, such that no individual outcome is "preferred" and dominates the rest

The prevailing thought in probability theory (frequentism) is that stabilizing relative frequencies are an objective phenomenon, independent of human thought. This assumption has served statisticians, casinos and insurance companies well. What this basically implies is that large sequences of similar random events are consistent and their averages can be confidently predicted within a "sufficiently" large sample.

Why can we predict the averages of big samples of individually unpredictable random events?

  • 2
    $\begingroup$ This seems to be a question about the philosophy of mathematics, and might also get good answers on Mathematics StackExchange. Since probability theory is central to science, it is also a question about the philosophy of science. I am not sure this is really a question about physics though. $\endgroup$ – Mark Mitchison Feb 3 '15 at 11:08
  • 1
    $\begingroup$ Yeah, I understand the point, and it is a very interesting question. Nevertheless, this is not "ObjectivePhenomena.StackExchange", for the very good reason that "objective phenomenon" is a slippery philosophical concept whose discussion bores the majority of physicists to death. In this particular case, I do not personally agree that "stabilising relative frequencies" should be called an objective phenomenon. It is a property of correlations between an observer and the observed system and depends on the model adopted. $\endgroup$ – Mark Mitchison Feb 3 '15 at 11:14
  • 2
    $\begingroup$ It looks as you are essentially asking why if an individual coin toss is random do we get the predictable result of 50% heads and 50% tails in the long term. Is that a fair summary? If so, this is the law of large numbers $\endgroup$ – John Rennie Feb 3 '15 at 11:23
  • 3
    $\begingroup$ Now it sounds as if you want a heuristic justification for the LLN, rather than the admittedly esoteric proof given in Wikipedia. If so, that's a challenge because it seems intuitively obvious to most of us. $\endgroup$ – John Rennie Feb 3 '15 at 11:45
  • 3
    $\begingroup$ This could be on topic at Statistics.SE as well as Math.SE. Either way, it's not really a physics questions. $\endgroup$ – Kyle Kanos Feb 3 '15 at 13:40

The single determining characteristic that is required for the emergence of the Law of large Numbers is that the various random events are independently random (or at least sufficiently so).

If I had a coin that I flip once, and then observe repeatedly, then those observations won't be independent. They'll be random, for sure: I cannot predict the result of the 2nd observation up front, but I can predict the result of all future observations after I observed the coin once. The LLN does not hold here.

But if I flip the coin as little as once every 1000 observations, then those observations are already sufficiently independent for the LLN to kick in. After all, the first observation now predicts only a minuscule fraction of the next million observations.

  • $\begingroup$ I guess it is just strange for me that large sequences of i.i.d events exhibit a predictable pattern consistently. If it cannot be deduced from other laws, I have to accept it as a basic property of randomness. $\endgroup$ – vantage5353 Feb 3 '15 at 12:20
  • $\begingroup$ @user1891836: The whole key is that there is no pattern ! A pattern implies repetition, which directly violates your "independendant" assumption. If there's a pattern of length 7, you do't need the law of large numbers; the number 7 is already large enough. $\endgroup$ – MSalters Feb 3 '15 at 12:23
  • $\begingroup$ Well, the pattern is not that you can predict the position of a certain outcome in a sequence, but that you can predict the distribution of outcomes in a sufficiently large sequence. After all, that's what casinos and insurance companies do. They don't know when an event will occur, but they expect it will occur at some time within a sample of a big enough size. $\endgroup$ – vantage5353 Feb 3 '15 at 12:27
  • $\begingroup$ @MSalters: I believe you're missing the point; I suspect the question sthat user1891836 wants to have answered is: Why is it legit to model a physical process like a coin toss as a random variable? Why should a series of such processes be treated equally (or at least all share an expectation value)?; the law of large numbers does not help with that as these are the assumptions that go into the theorem... $\endgroup$ – Christoph Feb 3 '15 at 12:59
  • $\begingroup$ @Christoph You put it way better than me! I am interested in precisely why it is reasonable that we assign equal probabilities to i.i.d events. Care for a chat? $\endgroup$ – vantage5353 Feb 3 '15 at 13:36

Toss a fair coin. For a single drawing the expected proportion of heads is distributed as follows (for $0$ and $1$):

$$\frac12, \frac12$$

For ten drawings in a row, by simple probability computation (from $0$ to $10$):

$$\frac{1}{1024}, \frac{10}{1024}, \frac{45}{1024}, \frac{120}{1024}, \frac{210}{1024}, \frac{252}{1024}, \frac{210}{1024}, \frac{120}{1024}, \frac{45}{1024}, \frac{10}{1024}, \frac{1}{1024}$$

The probability of very skewed drawings, like 10 heads in a row, goes decreasing with the length of the sequence, and the more "compliant" distributions are much more likely.

So completely deviating means are still possible, but with a smaller and smaller probability. The probability distribution always tend to concentrate around the mean. (The standard deviation is divided by the square root of the sequence length.)


Note that as empirical scientists, physicists don't really need any more than the ability to make testable predictions to justify a theory.

According to our current understanding, the most basic laws of nature (quantum theories) are probabilistic. As far as we know, that's just how the world works, and if there's a deeper reason behind it, we haven't been able to find it.

The prime example for a probabilistic system due to quantum effects would be radioactive decay.

At macroscopic levels, there are several ways for probabilities to emerge, and here are two ways I thought of:

First, we can have a family of interacting periodic systems. If you look at them only for a short time, they appear random; but if you waited for the least common multiple of the periods (or any time 'long enough'), they'll behave perfectly regularly.

A second example would be chaotic systems, which are effectively unpredictable - but this does not mean that you're equally likely to find the system in any particular region of phase space.

Now, what happens when you have a large number of such systems? Then, we have results like the law of large numbers or the central limit theorem. From a practical point of view, in many cases, you just end up either with random noise or a bell curve.

Now, let's look at a particular example: Tossing a coin.

Let's assume we're operating under ideal conditions, and the result of any toss only depends on two parameters: The coin's spin and its velocity.

The result will be periodic in both initial conditions, and if you overlay a source of noise, it's easy to see how you can end up with 50/50 probabilities if the parameters are right. The randomness of the coin toss got pushed back to the randomness of the noise.

  • $\begingroup$ Can you please elaborate on the "interplay of periodic and chaotic systems" and how this can lead to the emergence of stabilizing relative frequencies? $\endgroup$ – vantage5353 Feb 3 '15 at 11:57
  • $\begingroup$ @user1891836: There are no stabilizing frequencies. After a coin comes up head, further throws are still equally random which means tail hasn't become more likely. Of course, the average effect of that first single head is diluted as you flip more and more coins. But that's back to LLN. $\endgroup$ – MSalters Feb 3 '15 at 12:01
  • $\begingroup$ @MSalters I understand the Gambler's fallacy and that probabilities remain the same each trial. However, large samples of i.i.d outcomes exhibit a peculiar pattern: the averages tend toward the expected value, even though individual outcomes cannot be predicted $\endgroup$ – vantage5353 Feb 3 '15 at 12:06
  • 1
    $\begingroup$ @user1891836: the deviation from the average doesn't tend towards the expected value - actually it increases as $\sqrt{N}$. However the ratio of the deviation to the number of trials does tend to zero because it varies as $\sqrt{N}/N = 1/\sqrt{N} \approx 0$ for large $N$. $\endgroup$ – John Rennie Feb 3 '15 at 12:10
  • $\begingroup$ @user1891836: see edit $\endgroup$ – Christoph Feb 3 '15 at 12:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.