I can't understand how probability makes sense

Question

I have a lot of questions regarding probability. Please forgive me if I have made mistakes.

1. I actually tossed a coin 200 times. 54% of the time it landed on heads and 46% it landed on tails.

   What is the reason that there is a fair chance of the coin landing on heads or tails?

   Is it the randomness that causes this?

   If so, then in a purely random experiment would the result be 50-50?

   Even though probability only projects the likelihood of an event, why are the outcomes in favor of this projection?

2. If I eliminate all the external factors during a coin toss, like air resistance, the coin is tossed in a vacuum chamber, the force to flip the coin is fixed,etc will the experiment still be random? Or will I be able to predict the outcomes?

If the outcomes are indeed predictable, will the experiment be still random if I add a single atom into the chamber? If not at what point does it become random again?

Answer 1

j4nd3r53n's answer is great, and the starting point of mine.

What is the reason that there is a fair chance of the coin landing on heads or tails?

Probability can be thought of as a measure of uncertainty. When a person flips a coin in the real world, there are many, many factors that go into whether the coin flip will result in heads or tails: the force with which the flipper flips, the exact spot on the coin the force is centered, the exact vector of the force, the height difference between where it's flipped and stopped, deformations and imperfections in the coin, the wind, etc.... We say it's 50/50 because so many of those factors are wholly outside of the flipper's control and many that are theoretically within their control (eg., the force they use) are incredibly sensitive to tiny differences. Since humans out in the world can't choose which result they want, we have no information about which side will face up after the flip: we're completely uncertain, so neither choice is better than the other.

If I eliminate all the external factors during a coin toss, ... will the experiment still be random?

On the extreme other end, it is entirely possible to consider a machine that is designed to flip a coin just so so that it always lands heads-up, provided the coin can be placed into the mechanism correctly (eg., it might need to be heads-up to start, with the face angled just so). That machine could well flip the coin thousands of times and get heads each time. The difference here is that all of the factors that affect how the coin flies through the air are controlled, removing the uncertainty of the coin's trajectory. Or, more accurately, the variation now lives within the machine: whether the flipping arm malfunctioned or the case cracked or the landing pad wore down enough that the bounce is now "wrong". In removing sources of uncertainty, we've increased the probability of getting heads.

Is it the randomness that causes [a fair chance of the coin landing heads or tails]?

It's the other way 'round: the fair chance of the coin landing heads or tails is what results in the coin toss being random.

... in a purely random experiment would the result be 50-50?

In the real world, probably not: there's some evidence that heads will result about 51% of the time (partly due to the heads and tails sides not being evenly weighted, though there are other factors).

Here are the broad strokes of their research:

1. If the coin is tossed and caught, it has about a 51% chance of landing on the same face it was launched. (If it starts out as heads, there's a 51% chance it will end as heads).
2. If the coin is spun, rather than tossed, it can have a much-larger-than-50% chance of ending with the heavier side down. Spun coins can exhibit "huge bias" (some spun coins will fall tails-up 80% of the time).
3. If the coin is tossed and allowed to clatter to the floor, this probably adds randomness.
4. If the coin is tossed and allowed to clatter to the floor where it spins, as will sometimes happen, the above spinning bias probably comes into play.
5. A coin will land on its edge around 1 in 6000 throws, creating a flipistic singularity.
6. The same initial coin-flipping conditions produce the same coin flip result. That is, there's a certain amount of determinism to the coin flip.
7. A more robust coin toss (more revolutions) decreases the bias.

-- source

If the outcomes are indeed predictable, will the experiment be still random if I add a single atom into the chamber? If not at what point does it become random again?

Basically, the result will be as random as the unknowns or un-predictable-s allow/force it to be. With the coin flipper machine, a single atom is almost certainly not going to affect the path of the coin, but thermal expansion of the coin or the flipping arm might; a well-timed power surge or failure to the flipping arm is certain to add uncertainty to the flip.

I actually tossed a coin 200 times. 54% of the time it landed on heads and 46% it landed on tails.

For some more theoretical/math-ey bits, Wikipedia's Law of Large Numbers page has some good information and pointers. Extremely basically, it says that "the average of the results obtained from a large number of trials should be close to the expected value, and will tend to become closer to the expected value as more trials are performed." Which is to say, were you to keep flipping the coin, you'd get closer and closer to that 51/49 ratio. 200 trials is in the realm of a large number of trials, but it's on the low end.

This is not to be confused with the semi-serious Law of Truly Large Numbers, which states that "with a large enough number of samples, any outrageous (i.e. unlikely in any single sample) thing is likely to be observed" - flip the coin 1,000,000,000 times and getting 100 heads in a row becomes pretty likely.

Answer 2

This is a great question!

The randomness we talk about when we talk about the coin is not out there in the world. It is in our minds. We say the coin toss is 50/50 because we can't do better. If we could, we wouldn't say that it was 50/50. This isn't a paradox because probability is a fact about how we perceive the world, not about the world itself.

I wonder if this quote from Probability is in the Mind will be helpful:

To make the coinflip experiment repeatable, as frequentists are wont to demand, we could build an automated coinflipper, and verify that the results were 50% heads and 50% tails. But maybe a robot with extra-sensitive eyes and a good grasp of physics, watching the autoflipper prepare to flip, could predict the coin's fall in advance—not with certainty, but with 90% accuracy. Then what would the real probability be?

There is no "real probability". The robot has one state of partial information. You have a different state of partial information. The coin itself has no mind, and doesn't assign a probability to anything; it just flips into the air, rotates a few times, bounces off some air molecules, and lands either heads or tails.

Answer 3

I am in no way an expert in probability theory, but your question is about the understanding of what it actually is, rather than something more technical.

I think you are confusing probability and statistics (most people have problems with that, since they are so similar); the way I think about them is that statistics deals with the past: you carry out an experiment (toss a coin 200 times), then process the results. Probability, on the other hand, deals with the future: you are basically making the best possible guess about what is going to happen, based on what you know or assume.

The fundamental assumption is that if you get your starting conditions right, then your probability guess will be confirmed by your statistics; thus, in your experiment, you assumed that tossing a given coin would give heads 50% of the time, and it didn't after 200 tosses. The scientific method will tell you that your hypothesis was wrong somewhere, so you have to look for flaws that explain the outcome: was the coin 'perfect', was it the presence of air, was the sample of 200 tosses too small etc? Seeing that the deviation was fairly small, perhaps you need to continue for longer - 1000 tosses, perhaps?

To evaluate whether your result is too far out to just be a caused by a small sample size, you need statistical methods - something I know very little about. Perhaps somebody better equipped than me could help you there?

Answer 4

Even if the room is full of air molecules, wind, there's an earthquake happening, and the room is on a moving truck on a bumpy road, you could still in principle predict the results of the coin flips, because we live in a deterministic universe. Or, at best, whether or not we live in a deterministic universe is an unsolved and deep philosophical problem. In fact, if you get right down to it, it's hard enough to even define what "deterministic" really means. Thankfully, this has nothing to do with whether or not probability works, or what "random" means.

Do you know what a pseudo-random number generator is? This is basically a deterministic sequence of numbers $x_1, x_2, ...$ which "look" random. But they're still deterministic, often they're generated by choosing some initial $x_0$ (often the current time in milliseconds) and then applying some suitable recurrence relation $x_{n+1}=f(x_n)$. Totally deterministic.

And yet, the theorems of probability theory will still apply to this dataset. Why is that? It's because in the proofs of those theorems, we never use any hypothesis about where the data comes from. For instance, take the theorem that the expected value of a uniform random variable on $[0, 1]$ is $0.5$. You will find that this is true for your deterministic, pseudo-random sequence. The reason is because, expressed plainly, this theorem about expected value really says the following:

Suppose we have some large, finite sequence of numbers in $[0, 1]$. Suppose that for any interval $I\subseteq[0, 1]$, the proportion of numbers in the sequence in $I$ is about equal to the length of $I$. Then the average of the sequence will be about $0.5$.

Thus, the theorem will apply (approximately) to any large set of numbers which satisfies the stated assumption about proportions of elements in subintervals of $[0, 1]$. It doesn't require any constraints on the philosophical nature of the origin of your dataset.

In modern probability theory, the actual formulation of the theorem is this:

Let $P$ be a probability measure on $[0, 1]$ such that $P(I)$ is simply the length of $I$. Then the integral of the identity function on $[0, 1]$ with respect to the measure $P$ is $0.5$.

The reason this theorem applies to your sequence is because if $(x_n)$ is your sequence, you can define a measure on $[0, 1]$ by setting $\mu(A)$ to be the proportion of elements of $(x_n)$ in $A$. This measure approximately satisfies the above assumptions and therefore the theorem will approximately apply to it.$^1$

Now, why is it that probability applies to a physical die, then? It's because the laws of physics are a good pseudo-random number generator. If you were to write down all of the differential equations describing the motion of a die as you throw it, pick it up again, throw it again, etc, including all of the equations which describe the synapses firing in your body which cause you to throw the die one way and not another, you would presumably find that you could prove, as a mathematical theorem, that that dynamical system has the property that given any starting point, the sequence of results it will generate will consist of about one-sixths $1$-s, one-sixth $2$-s, and so on.

---

1. Before mathematicians settled on Kolmogorov's foundation modern probability theory, there was at least one competing theory which formulated probability more or less explicitly in the above terms: probability theory as the art of proving that if a finite data set has one kind of property, it must also have some other kind of property. This is the theory of "Kollektivs" of Von Mises.

Answer 5

Here’s a quick answer: The probability of getting exactly $100$ heads and $100$ tails in $200$ coin flips actually comes out as $$\frac{\binom{200}{100}}{2^{200}}\approx5.63\%.$$ So, not very “random”, is it?

However, the probability of getting within $10$ heads of $100$ is $$\frac{\sum_{k=90}^{110}\binom{200}{k}}{2^{200}}\approx86.3\%.$$

In words, probability tells you “something like this should happen”, but not “this will happen”.

Answer 6

If so, then in a purely random experiment would the result be 50-50?

Not likely.

The reason is because every toss is independent. The coin has no memory, so the result of a coin toss doesn't depend on previous tosses. Or if you're tossing 100 different coins, they don't know what the other coins are doing.

Supposing you plan on tossing the coin 100 times, and have a lucky streak and toss heads the first 50 times. There's no reason to expect that the next 50 will all be tails (in fact, you should probably check to make sure it's not a double-headed coin), or even that most of them will be. You normally expect any particular batch of tosses to be around 50-50, and this applies just as well to the second run of 50 as the entire run of 100.

As the number of tosses increases, we expect the final results to be closer to a 50-50 ratio, simply because there's nothing in the nature of the coin tosses that causes it to be biased in one direction or another (assuming a fair coin, and not using a mechanical tosser like minnmass describes). This is the Law of Large Numbers.

An important part of processing statistical data is determining whether there are dependencies in the way the data was generated.

Answer 7

You ask a natural science, or physical, question. Mathemathics has nothing to do with bumping into atoms. Not directly.

In the physical world, a coin can land and stay on the edge. In probability, 50:50 is not 49.9999:49.9999:0.00002. (Add two miles of nines and zeroes if you like, there is enough)

If so, then in a purely random experiment would the result be 50-50?

In the long run, yes. But it could even fluctuate, so it would be 50:50 overall, but still not random. You should do 10 times 20 tosses, and compare. Or even 100 times 2 tosses to see the boring 4 HH, TT, HT and TH combinations: you get some double heads, some double tails, and same amount of mixed. (see added bottom part)

Don't polish your coin or toss it high, hoping for a 50:50 guarantee - you only have a "mostly somewhere around 50:50" guarantee, and it contains "mostly".

You know quite well what a "long enough run" is: after ten oder twenty events you get a good impression, and you will even clearly note the difference between a sequences like 0000011111 and 0010111010.

You would have to plot each head or tail over time, and then use some statistical tests to look for non-random distributions.

Something outside the "confidence" interval then needs re-evaluation, if you want to ne sure(r).

---

In the end, you can not know: say you toss a coin one hundred times and really get 100:0! You repeat. Normal, say 47:53.

You may never find out, why you got 100:0 in one of the "runs". The wind? Rather not. You tossed correctly. A magnetic field? Or was it really coincidence? Do you have witnesses?

At least mathematics can tell you how very unlikely a 100:0 coin toss is, if (and only if) it was "pure" random.

After you hear the probability, you would be very very confused. "I tossed 100 times and got 100 heads. Nobody even saw it. I didn't even bet on it." Luckily, this never happens.

Ever.

Or does it?

(maybe it happens in catastrophies and miracles, but not in statistical experiments)

---

Back to 50-50, and pascal triangle. As a complete "decision" tree, a repeated (or multi-) coin toss can be written like this:

First toss/coin: 0 or 1 Second: 0, then (0 or 1) or 1, then (0 or 1)

--> Two coins can only come in 4 different combinations: 00, 01, 10, 11

Since "01" and "10" is the same (order does not matter here) you lump them together and get: one "00", two "mixed", one "11".

Or: two mixed and two pure.

--> with only two 50-50 events you have also a 50-50 chance of getting 100:0 or 50:50.

With three coins you reach the 1-3-3-1 line in pascal triangle:

00 and 1 or 0
01 and 1 or 0
10 and 1 or 0
11 and 1 or 0

000, 001; 
010, 011; 100, 101
110, 111;

Now you have 8 combinations, and you see the pattern emerging:

"000" has exactly the same probabllity than say "100", but: "100" has companions: "001", "010". These are the three with exactly one "1":

no  "1": 000
one "1": 001, 010, 100
two "1": 011. 101. 110
all "1": 111

2 out of 8 are pure ("000" and "111"), 6 are mixed (but none can even be 50:50, only together do they contain 9 "1" and 9 "0").

Next pascal triangle row is 1-4-6-4-1. That will mean 6 out of 16 four-coin tosses will show exactly 2 heads and 2 tails. 8 out of 16 tosses will have some unequal mix of head and tails, and 2 tosses will be purely heads or purely talis.

--> With 4 coins it is rather unlikely (under 37%) to get a 50-50 result.

With 4000 coins it gets even more unlikely to hit that single "center column" exactly. Left and right to it, there are so many "companions", that it is much more probable to get "something like" 50.103:49.897 than that single 50.000:50.00 column,

If you find the pascal triangle unpractical for your 200 tosses, you might want to turn to gauss curve ("distribution" curve) - but the core of this all ("variance") starts with pascal triangle.

  x
 xxx

 xx 
 xx
xxxx

  x
  x
 xxx
 xxx
 xxx
xxxxx 

If you want exactly Heads=Tails, then you better use exactly 2 coins: in about 50% of tosses you will get exactly 50.0% Heads. (but "failure" will be a full 100% or 0% failure)

The more coins, the more likely it gets to come close to 50.0%, but at the same time more unlikely to hit 50.0% exactly. A 100% or 0% result gets very unlikely.

A kind of paradoxon of large numbers, but we all know:

"it is not normal to be (exactly) normal".

Answer 8

I suppose you are talking about the hypothetical situation of a fair coin so I'll talk about the probabilities of what to expect with a fair coin. If you're going to flip a coin 100 times, before you start, the probability of every outcome is exactly 0.5. After you flip it 100 times, you gain information of what the actual outcome was. If you roll a die just once, after you roll it, you wouldn't say the probability given what you observed is $\frac{1}{6}$ for each side of the die. Now if you decided ahead of time that you were going to tell somebody else only how many times the coin landed on heads and how many times you flipped it but not what it landed on each time and then it didn't land on heads exactly 50 times, now for that person, the probability of what any single outcome was is not exactly 50/50.

The probability distribution of how many times the coin will land on heads is determined by the Pascal triangle. If you take the Pascal triangle and then for each nonnegative integer $n$, you divide all the entries in the $n + 1^{st}$ row by $2^n$, then as you go down the rows, you get something resembling the diffusion of heat. The math shows that as $n$ gets larger, the probability distribution of how many times the coin will land on heads if you flip it $n$ times is a normal distribution with a standard deviation of $\sqrt{n}$. You can expect that with a really large number of flips, the fraction of times it lands on heads is very close to 0.5.

Answer 9

You forgot to take the variance into account.

The variance, naively speaking, is the expected value of the square of the difference between what you actually get and what you expected to get. Its square root, the expected value of the actual difference, is the standard deviation.

For a fair coin, the expected value, or mean, is in fact 1/2 (assigning 1 to heads and 0 to tails (or vice versa)). It's calculated by multiplying the value of each result by its probability of occurrence and then summing everything up. In this case: $$\mu = E(X) = \sum x \cdot P(X=x) = H \cdot .5 + T \cdot .5 = 1/2 + 0/2 = 1/2$$ Again, we say $H=1$ and $T=0$ to make the math work.

The variance is then calculated by finding $E((X-\mu)^2)$, which, after some math, can be more easily calculated as $E(X^2) - E(X)^2$, that is, the expected value of the squares minus the square of the expected value. In this case: $$\sigma^2 = V(X) = E(X^2) - E(X)^2 = H^2 \cdot.5 + T^2 \cdot .5 - (1/2)^2 = 1/2 +0/2 -1/4 = 1/4$$ Taking the square root to get the standard deviation, we find that $\sigma = 1/2$, which makes sense: $\mu-\sigma = 1/2 - 1/2 = 0 =T$ and $\mu+\sigma = 1/2+1/2=1 = H$. For one coin, we expect to get either a head (1) or a tail (0) on any single flip.

Now, for many flips, the Law of Large Numbers applies. This states that $n$ trials of the same experiment should, taken as a single experiment, have a mean equal to the sum of each trial's mean (that is, $\mu_{Total} = n\mu$) and a variance equal to the sum of each trial's variance (that is, $\sigma_{Total}^2 = n \sigma^2$). For 200 flips, we should expect $200 \cdot 1/2 = 100$ heads on average, and we should expect within $\sqrt{200 \cdot 1/4} = \sqrt{50} \approx \pm 7.07$ heads from that average on any given 200 flips.

In your experiment, you got $56\% \cdot 200 = 108$ heads, which means you got 8 extra heads than expected. Your experiment is a little anomalous, but not unexpectedly so. Why? Because variances have their own variances - they are expected values of random variables after all.

Answer 10

First of all, when doing mathematical probability we have only a theoretical model of how the coin toss works. The simple theoretical models of tossing coins and rolling dice that you find in a typical introductory course on probability are good enough to make lots of money for casinos that run craps tables, but they don't absolutely guarantee that reality will follow the theory. Perhaps it is possible for someone to become skilled enough that they can flip a supposedly "fair" coin in a way that gives a significantly higher chance of their preferred outcome, or roll a pair of dice so that some results are much more likely than others.

Your experiment with the coin, however, matches the theoretical model of a fair coin so well that the reality of your experiment is practically indistinguishable from the theory. Specifically, $108$ heads out of $200$ tosses is a bit higher than expected from a fair coin, but not so much higher that you ought to seriously believe that the coin is biased toward heads in the absence of any other evidence. In fact, if your coin tosses were completely random, unbiased, and independent of each other, there would be approximately a $14.44\%$ chance to get at least $108$ heads in $200$ tosses. There is almost a $28.9\%$ chance that the percentage of heads would be at least $54\%$ or that the percentage of tails would be at least $54\%.$ So concluding after your experiment that the coin toss is biased would be like rolling a die once, observing that it came up $6$, and concluding that the die is biased toward high numbers.

It is paradoxical, but if a sequence of trials is truly random and each trial is independent of the others, then the ratios of frequencies of events over a series of trials usually do not exactly match the ratios of probabilities of the events. If you could guarantee that the observed ratios will match the expected ratios, it would mean either that the trials were not random or that they were not independent; there would be circumstances when you could predict the next outcome based on the previous outcomes.

For example, suppose (for argument's sake) that a $50\%$ probability of heads guaranteed that in any long run of tosses there would be exactly $50\%$ heads (or as near $50\%$ as possible). So in $200$ tosses you would see exactly $100$ heads. Well, if $200$ tosses is enough tosses to guarantee a $50\%$ outcome, $202$ tosses also is enough. So we could commence on a sequence of $202$ tosses, and after $201$ tosses you would have either $100$ heads or $101$ heads and you would know for sure whether the next toss was going to be heads (because you would know that you would have $101$ heads after that toss).

We might get around that example by saying the you can have two more heads than tails, but not more than that. But then what about a series of $204$ tosses where you get $102$ heads in the first $202$ tosses? Then you know the next toss has to be tails. So we need it to be possible for there to be three more heads than tails, even if the coin is completely unbiased. And that leads to needing it to be possible that there are four more heads than tails. In fact, if we toss the coin sufficiently many times, we should expect eventually to see the number of heads lead the number of tails by any amount you want. Not in spite of the unbiased randomness of the toss, but because of it.

But what about the percentage difference? Can we at least put an absolute bound on that? Again, no, because that would imply that you could find yourself in a situation where the percentage of heads was already at the maximum possible value, or near enough that one more head would put you over the maximum, and you would know that the next toss had to be tails. What we can say about the percentages, however, is that certain percentages are very unlikely to occur, and increasing the number of tosses makes them even less likely to occur. Still, $54\%$ within $200$ tosses isn't particularly surprising. Now $54\%$ within $2000$ tosses (probability $<0.02\%$ given a fair coin) or $64\%$ within $200$ tosses (probability $<0.005%$ given a fair coin) would be quite surprising and would justify looking for a "trick" that is causing the coin to come up heads more often.

Answer 11

Randomness is not always 50-50. For example, the chance of drawing an ace from a shuffled deck is 4/52. The chance of rolling a 6 with a single die is 1/6. Randomness is related to unpredictability, which generally comes from a lack of exact information on the spin, velocity, position, and orientation of the coin's launch and how it responds to spin, gravity, air currents, air resistance, and contact with the surface it lands on.

Mathematicians have found that probability is a good description of coin tossing observations.

In physics there is a very deep theory called quantum mechanics in which everything is defined in terms of probability waves. Thus probability is at the core of everything that exists. Be patient and you will eventually get to this.

Challenge: you are working for one of our Nevada casinos, and your boss asks you to test a die to see if each face has a probability of 1/6. What do you do? What if you must accept or reject a shipment of 10,000 dice?