If a coin toss is observed to come up as heads many times, does that affect the probability of the next toss?

Question

A two-sided coin has just been minted with two different sides (heads and tails). It has never been flipped before. Basic understanding of probability suggests that the probability of flipping heads is .5 and tails is .5. Unexpectedly, you flip the coin a very large number of times and it always lands on heads. Is the probability of flipping heads/tails still .5 each? Or has it changed in favor of tails because the probability should tend to .5 heads and .5 tails as you approach an infinite number of trials?

I understand that flipping coins is generally a stochastic process, but does that change at all if you see a large number of trials bias to one side?

Answer 1

If you don't know whether it is a fair coin to start with, then it isn't a dumb question at all. (EDIT) You ask if the coin will be biased towards Tails to account for the all of the heads. If the coin was fair, then the answer from tilper addresses this well, with the overall answer being "No". Without that assumption of fairness, the overall answer becomes "No, and in fact we should believe the coin is biased towards heads.".

One way to think about this is by thinking of the probability $p$ of the coin landing heads to be a random variable. We can assume that we know absolutely nothing about the coin to start with, and take the distribution for $p$ to be a uniform random variable over $[0,1]$. Then, after flipping the coin some number of times and collecting data, we change our distribution accordingly.

There is actually a distribution which does exactly this, called the Beta Distribution, which is a continuous distribution with probability density function $$f(x) = \frac{x^{\alpha-1}(1-x)^{\beta-1}}{B(\alpha,\beta)}$$ where $\alpha-1$ represents the number of heads we've recorded and $\beta-1$ the number of tails. The number $B(\alpha,\beta)$ is just a constant to normalize $f$ (however, it is actually equal to $\frac{\Gamma(\alpha)\Gamma(\beta)}{\Gamma(\alpha+\beta)}$).

The following graphic (from the wikipedia article) shows how $f$ changes with difference choices of $\alpha,\beta$:

As $\alpha \to \infty$ (i.e. you continue getting more heads) and $\beta$ stays constant (below I chose $\beta=5$), this will become extremely skewed in favor of $p$ being close to $1$.

Answer 2

You have wandered into the realm of Bayesian versus frequentist statistics. The Bayesian philosophy is most attractive to me in understanding this question.

Your "basic understanding of probability" can be interpreted as a prior expectation for what the distribution of heads and tails should be. But that prior expectation actually is itself a probabilistic expectation. Let me give some examples of what your prior expectation (called the Bayesian prior) might be:

1) Your prior might be that the frequency of heads is exactly $0.5$ no matter what. In that case, no number of consecutive heads or tails would shake that a priori certainty, and the answer is that your posterior estimate of the distribution is that it is $0.5$ heads.

2) Your prior might be that the probability of heads is a normal distribution with mean $0.5$ and standard deviation $0.001$ -- you are pretty sure the coin will be pretty close to fair. Now by the time the coin has landed on heads 100 consecutive times, the posterior estimate of the distribution is peaked sharply at around $0.995$. Your Bayesian prior has allowed experimental evidence to modify your expectation.

The alternative approach is frequentists. That says "I thought (null hypothesis) that the coin was fair. If that were true, the likelihood of this result of no tails in a hundred flips is very small. So, I can reject the original hypothesis and conclude that this coin is not fair.

The weakness of the Bayesian approach is that the result depends on the prior expectation, which is somewhat arbitrary. But when a lot of trials are involved, an amazing spectrum of possible priors lead to very similar a posteriori expectations.

The weakness of the frequentist approach is that in the end you don't have any expectation, just the idea that your null hypothesis is unlikely.

Answer 3

Is the probability of flipping heads/tails still .5 each?

If you already had the assumption that the coin is fair, then yes. Although statistically it's highly unlikely to flip heads a "very large number of times" in a row, the probability will not be changed by past outcomes. This is because each coin toss is independent from all other coin tosses.

Or has it changed in favor of tails because the probability should tend to .5 heads and .5 tails as you approach an infinite number of trials?

Already answered above but here's more detail. The law of large numbers (LLN) says that if we flip a fair coin a bunch (thousands, millions, billions, etc.) of times in a row, the outcomes should be approximately half heads and half tails. But LLN doesn't tell you anything about the probability of the next (or any) trial.

Answer 4

Probability is a mathematical model. You need to separate the modeling process from the application of the model.

We can model a coin toss by a two-element probability space {H,T} with a probability of 0.5 assigned to each element. If you sample 100 times from this probability space, and get 100 heads, then the probability of a head on the next sampling is 0.5. The probabilities are assigned in the set-up of the model and they don't change.

The question can therefore be rephrased from "Is the probability of flipping a Head 0.5?" to "Is the model appropriate to this coin?" This allows you to separate the purely mathematical part of this from the modelling process.

To answer the modelling question a Bayesian approach would make the most sense to me. You would need to quantify your confidence in you prior assumption (that a 0.5:0.5 model is appropriate) in some way. Since different people would have different levels of confidence, it is natural that different people would end up with different opinions about the appropriateness of the model. The Bayesian approach allows one to quantify what Carl Sagan meant by "Extraordinary claims require extraordinary evidence".

Answer 5

While we know that the proportion of heads we get from flipping a fair coin should be about half, it's worth looking at the expected amount of deviation:

If you flip a coin a million times, it's fairly rare to get exactly half a million heads: that happens with less than 0.1% probability! Even having the error be 100 or less is uncommon: that happens with about 16% probability.

In fact, if you flip a coin $n$ times, on average the the average amount of error* you get is around $\sqrt{ \frac{n}{2 \pi} }$.

So, suppose you flip a fair coin 100 times in a row and got heads every time — a surprising result saying you got 50 extra heads than average. If you then continued to flip it a million more times... by that point the average size of the error is around 1596, so that extra 50 flips isn't noticeable at all anymore.

So if you get a surprising number of heads early on, the coin has absolutely no need to be skewed towards flipping tails in order for the future statistics to behave like a fair coin usually does.

*: More precisely, this is (approximately) the expected value of $|H - \frac{n}{2}|$, where $H$ is the number of heads.

Answer 6

This is a great question that digs at the heart of how we use probabilities. There are several great answers, but I'd like to offer one which is more intuitive in nature.

The key to understanding the issue you are dealing with is understanding your assumptions. Let us say that you have a coin which, when flipped, lands heads 50% of the time and tails 50% of the time. This is a fair coin. You flip this coin 100 times and it lands on heads every time. It still has a 50/50 chance of landing tails on the 101th toss. That's simply how probability works.

Now let's explore your case. In your case you have a coin which has never been flipped. You then state

Basic understanding of probability suggests that the probability of flipping heads is .5 and tails is .5.

This is an assumption. You are tying the real life coin to a statistical distribution, assuming you understand how it will behave. You did not state why you feel this assumption is justified, but presumably this is because you have some experience with the physics of how objects called "coins" tend to behave when flipped. You may even have a mathematical model based on a theory of how the material in the coin is distributed.

You now flip the coin 100 times. What was the expectation for the number of times it lands on heads? Well, we don't actually know. However, we have made an assumption that this flipping process will yield 50/50 odds, so you expect 50 heads. You can also then use the Central Limit Theorem to determine the variance you expect, which will be quite narrow for 100 tosses.

Then your coin actually lands on heads 100 times. What now? Nothing in the probabilities stated a fair coin couldn't land on heads 100 times. Indeed, probability suggests that one should expect that occurrence. It would be rare, but probability would state that it should happen now and then.

You made an assumption earlier: that the coin was fair. This was justified in some method, typically by an assumption that you know how coins are flipped. Rationally, we want to think of these assumptions as sacrosanct, but intuitively you know something's going on (after all, you asked a question on Mathematics.SE about it!). So let's take a rational look at it. We made an assumption, perhaps that assumption was wrong. Perhaps the coin is magnetic and we are in a magnetic field. Perhaps the coin is weighted strangely. Perhaps its just dumb luck and our assumption was actually valid.

At this juncture, statistics and probability provide two major approaches for dealing with this rational conundrum: frequentist and Bayesian. Both of these schools have good answer with lots of points on this question. The frequentist approach is to explore whether we should reject the assumption that the coin was fair to begin with. One gathers data points (such as the 100 tosses) and then makes decisions. One may decide that the probability of 100 heads coming from a fair coin from random chance is low enough that you wish to reject the hypothesis that the coin was fair in the first place. That wording is careful. It doesn't state that the coin is fair, merely that it rejects an assumption that it was fair because the data challenges the assumption.

The Baysians take a different approach, and instead try to continuously improve their assumptions. They start with the assumption that the coin is fair. Perhaps more precisely they assume that the actual distribution is very close to that of a fair coin. They then begin flipping the coin in the test. Each time they update their assumptions about the probability distribution of the coin. As they do, they develop a hypothesis that the coin is more and more unfair as they see more and more heads.

Both approaches have their merits. There's people in each camp. However, the thing they both agree upon is that the assumption that the coin is fair is an assumption worth challenging. The frequentists try to reject the hypothesis, the Bayesian try to adapt it into a more correct hypothesis, but neither camp tries to hold themselves to the assumption that the coin is fair.

Because the world isn't fair.

Answer 7

Basic understanding of probability suggests that the probability of flipping heads is .5 and tails is .5

If that is what you believe, then I have a coin here of uncertain provenance and a wager you might be interested to undertake. I will of course call heads or tails, all you are required to do is put down your $100. Suddenly you aren't so confident it's a fair coin. And that's without even tossing the coin once, I was just talking

Not knowing anything about a coin is not sufficient grounds to assume that it's a fair coin[*]. There is a sense in which we might believe that an unknown coin has a 50% chance of coming up heads, and that's if we believe that of all the unfair coins in the world, probably half of them are biased towards heads and the other half equally biased towards tails. I've no idea whether that's true or not, but if it is then we would say that a coin selected uniformly randomly from all the coins in the world has a 50% chance of coming up heads, but that the conditional probability of such a coin producing a second head after coming up heads on the first toss is more than 50%.

The issue here is that before assigning a probability you need to have a model of the process that produces the outcomes you want to assign probabilities to. Just saying "it has to be either X or Y, so let's assume the probability is 50/50" might sound vaguely plausible for coins because most coins are fair or very close to it, but not all. Without making any assumptions about where this coin came from we can't come up with a probability.

has it changed in favor of tails because the probability should tend to .5 heads and .5 tails as you approach an infinite number of trials?

Not unless we believe the coin has a "memory" of its past throws, which seems physically implausible if it's just a flattish lump of metal. Otherwise the probability we assign to another head after several heads should be either 0.5 if we have convincing grounds to believe the coin to be fair (which, if it's a real coin made by a reputable mint, we do), or some larger number if we've assigned a conditional probability that it is an unfair coin, given its past performance. The latter is pretty reasonable given that we know unfair coins exist (or, more realistically, unfair coin tosses). Actually quantifying the likelihood that we're currently encountering one, requires some assumptions about the gentleman who has just taken $100 off our hands several times in a row.

Note that as you tend to an infinite number of trials, the ratio of heads to tails of a fair coin approaches 1. However, the ratio of "the number of heads plus one million" to the number of tails, also approaches 1. That is why a fair coin doesn't need memory in order to be fair. If the first few throws are all heads then it doesn't need to somehow "make up for them" later, because as you approach infinity the first few results approach irrelevant. Every good throw of a fair coin is 50/50.

The concept in experimental science of using a p-value to reject a null hypothesis is, in effect, saying, "this coin has come up heads so many times in a row that I will no longer entertain the hypothesis that it is a fair coin".

[*] - it's effectively impossible to make a coin so biased that it has a good chance of coming up heads over and over again when thrown well. That is, barring a double-headed coin, or a "coin" with some peculiar profile that we wouldn't accept really is a coin. But if it's allowed to fall to the ground I believe non-trivial bias is possible, and if the con-artist flips it then we're in business. Mathematics lessons contain more problems about biased coins, than suggestions how to actually make one that looks anything like a normal coin, so you need to be more wary of biased coins in fake problems than in real life. But you can just expand the subject a little -- not whether the coin is fair, but whether your technique for tossing it is fair.

Answer 8

If a coin came up heads 1000 times in a row, I'd suspect that it was NOT a fair coin. Maybe both sides are heads, or the tails side is heavier, or there are hidden magnets involved, or some other magic trick. If I saw a coin come up heads 1000 times in a row, I'd bet that the next toss will be heads again, not tails.

But assuming that we somehow know that it really is a fair coin: One flip cannot affect the next flip because the coin has no memory. How would it "know" that it had come up heads 1000 times so now it's "due" to come up tails? Unless there's electronics hidden inside the coin recording each result and affecting future results.

Suppose you tossed a coin but didn't let me see which way it turned up. Now I toss the coin. Are the results of my toss affected by the result I didn't see? What if today I toss a coin 10 times and it comes up heads all ten times. Then I don't toss it again until tomorrow. Do yesterday's results affect today's toss? What if I don't toss it again for a week or a month? What if I never toss it again in my life, but I leave it to my son, and he leaves it to my grandson, and a hundred years from now somebody tosses it again. Will that toss be affected by the result a hundred years before?

All coins should have a 50/50 chance of coming up heads. What if you toss a coin and it comes up heads 100 times in a row, and then I toss a different coin? Is it more likely to come up tails to make up for your coin? What if someone in the next room gets heads 100 times in a row? Someone on another continent?

Suppose I have some trick to make the coin come up heads. Like, I've figured out how to toss the coin in just the right way so it flips over in the air exactly twice and comes down with the same face up that it had before the toss. Does "cheating" like this affect future fair tosses?

Etc. While you may think that intuitively, if it comes up heads a few times in a row it's now "due" to come up tails, the more you think about it the less sense this makes. There's no way for one toss to affect the next.

Lady Luck has permanent amnesia. A truly random event is, by definition, not affected by previous events.

Answer 9

A given process has a certain probability distribution. This distribution does not change due to the fact that you have performed that process finite many times and got a sequence of results that is very unlikely (unless you alter the process in that way).

In principle you can only get a knowledge of this probability distribution if you perform the process infinite many times, which isn't possible. Think about it like this: Any finite unlikely sequence can be evened by an infinite number of results that come after it. Compared to infinity, any finite result is negligible!

This is why an experimental model based on some experimental data (e.g. the finite sequence of results) is always given with respect to a certain confidence level. This confidence level represents a likelihood that the model is correct (e.g. can make predictions) based on that sequence of results.

Suppose your model is that your coin is a fair coin and as such obeys the binomial distribution: the number $k$ of heads within a sequence of $n$ measurments has a probability of $p_{k,n} = \binom{n}{k} \cdot 0.5^n$ to occur. Say you perform $n=100$ measurments and get $k=100$ heads. The probablity for that is $p_{100,100} \approx 7.89 \times 10^{-31}$. Pretty unlikely. However in 1 out of $1/p_{100,100} \approx 1.27 \times 10^{30}$ cases you would be correct to say that your coin is a fair coin. So in 1 out of $1.27 \times 10^{30}$ cases the next flip would have equal probability to show head or tail. In $1.27 \times 10^{30} - 1$ cases the coin would be pretty unfair though.

The problem here is, that you cannot know with certainity whether or not your coin is a fair coin. Now you can choose to standpoints: You can either say that

1. your coin is a fair coin with a confidence level of $p_{100,100}$ or
2. your coin is unfair with a confidence level of $1 - p_{100,100}$.

The confidence level gives the likelihood of your statement to be correct based on your performed experiment.

If you choose the second standpoint your expected probablity distribution has changed significantly from fair to unfair. The real probablity distribution is, however, unchanged. The coin will continue to be fair or unfair regardless of what you say.

Answer 10

There is a remote possibility that a coin could be made where the result of one throw would influence the result of the next throw. Well, just make a coin that weighs 50kg and it will probably come up the same way all the time :-)

Ignoring that possibility, the probability for that coin for throwing heads or tails doesn't change. However, each throw gives you some information about the probability that the coin is biased, and which way. I have in my drawer five coins, which will come up heads with probabilities 0, 1/4, 1/2, 3/4 and 1, respectively. You take a random coin out of my drawer. The probability of getting heads in your first throw is 1/2, but you don't know the probability for the particular coin you took. But if you had five throws and five times heads, it's highly likely that you picked the coin with 3/4 or 1 probability. If you have 95 heads in 100 throws, it is close to certain that you picked the coin with 3/4 probability and by unlikely coincidence got lots of heads.

Answer 11

I think the question should have been put differently: Under the assumption that there is a fixed but unknown probability of heads $p$ can one predict or estimate its value with some degree of accuracy based on the results of the previous tosses(our data).

Now, the LLN says that under specific conditions when an experiment is repeated ad infinitum the empirical average of the outcomes comes very close and with great certainty to the theoretical average ie the mean value of the random outcome. In the case of the coin it means that if the trials are independent and the (unknown) probability of heads remains constant throughout the experiment then the frequency of appearances of heads approaches very closely and with great certainty the theoretical(and still unknown) probability of heads.

So based solely on the data one can say that the probability of heads is approximately the frequency of appearances of heads. If we see countless heads in a row and we accept the above assumptions we would be almost certain that the next one will be head.

Now if one is Bayesian, wants to play with $p$ and puts a prior uncertainty on that then one must calculate a quantity called posterior predictive probability ie Prob(head in the next trial| data) where data=the results of all previous tosses.

But as Bayesian estimates are a combination of empirical(data) and prior information when the size of your sample is very large the empirical part is dominant so I would say that the frequency of heads in a infinite number of tosses is again a safe estimate of the probability of heads.

Answer 12

Also, to play devil's advocate, remember there is also the Gambler's Fallacy theorem.

"I will wager a bet on either red or black. There is a 1 in 2 chance of either outcome. There has been 49 red, so the next outcome must be black!"

Each incidence is a specific, discrete event that is unrelated to the other (all other factors considered, in relation to the elimination of bias, undue influence and assumption of all variables equal across each occurence of the spin of the roulette wheel).

In situations were the outcome is truly random, the assumption of balancing is erroneous and fallacious logic.