A two-sided coin has just been minted with two different sides (heads and tails). It has never been flipped before. Basic understanding of probability suggests that the probability of flipping heads is .5 and tails is .5. Unexpectedly, you flip the coin a very large number of times and it always lands on heads. Is the probability of flipping heads/tails still .5 each? Or has it changed in favor of tails because the probability should tend to .5 heads and .5 tails as you approach an infinite number of trials?

I understand that flipping coins is generally a stochastic process, but does that change at all if you see a large number of trials bias to one side?

  • 11
    $\begingroup$ This is an important question that has sparked the interest of several mathematicians in the past. Laplace had a similar question that is now called the sunrise problem: "What is the probability that the sun will rise tomorrow?" (given we have seen it rise all previous days) en.wikipedia.org/wiki/Sunrise_problem $\endgroup$
    – Michael
    Jul 18, 2016 at 18:44
  • 8
    $\begingroup$ The issues here are: (i) Can we assume coin flips are independent and identically distibuted (i.i.d.) with some fixed probability $p$? (ii) If so, can we assume we have an a-priori distribution on $p$? (i.e., is $p=1/2$ with prob 1, or is $p$ uniformly distributed over $[0,1]$, or what?) If "yes" to both, then we have a well defined probability problem. If "no" to both then we have to somehow bootstrap a reasonable probability model from our observations, and then make claims with respect to this model. $\endgroup$
    – Michael
    Jul 18, 2016 at 18:51
  • 25
    $\begingroup$ I think the Op is asking a simpler question than the answers suggest. Isn't it just a case of the Gambler's Fallacy? $\endgroup$
    – PeteGO
    Jul 18, 2016 at 19:47
  • 5
    $\begingroup$ @hardmath "e.g. that it has two sides." Erm, even in its original version the question specifies that the coin is two-sided. "You state this without offering any evidence..." What would it mean to "offer...evidence" for the hypothetical problem-constraint? $\endgroup$ Jul 18, 2016 at 20:02
  • 7
    $\begingroup$ Repeated heads does not make it more likely the next flip will be tails. If it makes any difference then it makes it more likely to be heads $\endgroup$
    – Henry
    Jul 18, 2016 at 20:32

12 Answers 12


If you don't know whether it is a fair coin to start with, then it isn't a dumb question at all. (EDIT) You ask if the coin will be biased towards Tails to account for the all of the heads. If the coin was fair, then the answer from tilper addresses this well, with the overall answer being "No". Without that assumption of fairness, the overall answer becomes "No, and in fact we should believe the coin is biased towards heads.".

One way to think about this is by thinking of the probability $p$ of the coin landing heads to be a random variable. We can assume that we know absolutely nothing about the coin to start with, and take the distribution for $p$ to be a uniform random variable over $[0,1]$. Then, after flipping the coin some number of times and collecting data, we change our distribution accordingly.

There is actually a distribution which does exactly this, called the Beta Distribution, which is a continuous distribution with probability density function $$f(x) = \frac{x^{\alpha-1}(1-x)^{\beta-1}}{B(\alpha,\beta)}$$ where $\alpha-1$ represents the number of heads we've recorded and $\beta-1$ the number of tails. The number $B(\alpha,\beta)$ is just a constant to normalize $f$ (however, it is actually equal to $\frac{\Gamma(\alpha)\Gamma(\beta)}{\Gamma(\alpha+\beta)}$).

The following graphic (from the wikipedia article) shows how $f$ changes with difference choices of $\alpha,\beta$:

PDF of Beta Distribution for different parameters

As $\alpha \to \infty$ (i.e. you continue getting more heads) and $\beta$ stays constant (below I chose $\beta=5$), this will become extremely skewed in favor of $p$ being close to $1$.

enter image description here

  • 1
    $\begingroup$ @Michael I created a new plot, as requested. Ended up going with a fixed $\beta$ and going with larger $\alpha$s. $\endgroup$
    – Hayden
    Jul 18, 2016 at 19:41
  • 6
    $\begingroup$ Why is a newly-minted coin fair? Magic shops sell coins that have been specifically minted to not be fair, so I don't think that one can simply assume that 'newly-minted' means fair. In the case where it is fair, then there are other answers here which address that. $\endgroup$
    – Hayden
    Jul 18, 2016 at 20:03
  • 6
    $\begingroup$ @AaronHall Having two sides doesn't mean the coin is "fair". It could be weighted or magnetized in such a way as to favor a particular side. $\endgroup$ Jul 18, 2016 at 20:03
  • 4
    $\begingroup$ @Max If you truly know nothing about the coin to start with, then it is reasonable to use a uniform distribution, but there is certainly nothing telling us what our initial distribution should be. The benefit of using the uniform distribution to start is that the only information going into our updated distribution is the results of the flips we've done. If you have some other initial distribution then you could update that one by using a posterior distribution. $\endgroup$
    – Hayden
    Jul 18, 2016 at 22:10
  • 3
    $\begingroup$ @max I didn't mean anything rigorous by that, only that if we had some idea of where $p$ was, then our initial distribution wouldn't be uniform. A uniform distribution (on $B$) does not favor any particular region of $B$ for any reason aaide from the measure of that region. $\endgroup$
    – Hayden
    Jul 18, 2016 at 23:22

You have wandered into the realm of Bayesian versus frequentist statistics. The Bayesian philosophy is most attractive to me in understanding this question.

Your "basic understanding of probability" can be interpreted as a prior expectation for what the distribution of heads and tails should be. But that prior expectation actually is itself a probabilistic expectation. Let me give some examples of what your prior expectation (called the Bayesian prior) might be:

1) Your prior might be that the frequency of heads is exactly $0.5$ no matter what. In that case, no number of consecutive heads or tails would shake that a priori certainty, and the answer is that your posterior estimate of the distribution is that it is $0.5$ heads.

2) Your prior might be that the probability of heads is a normal distribution with mean $0.5$ and standard deviation $0.001$ -- you are pretty sure the coin will be pretty close to fair. Now by the time the coin has landed on heads 100 consecutive times, the posterior estimate of the distribution is peaked sharply at around $0.995$. Your Bayesian prior has allowed experimental evidence to modify your expectation.

The alternative approach is frequentists. That says "I thought (null hypothesis) that the coin was fair. If that were true, the likelihood of this result of no tails in a hundred flips is very small. So, I can reject the original hypothesis and conclude that this coin is not fair.

The weakness of the Bayesian approach is that the result depends on the prior expectation, which is somewhat arbitrary. But when a lot of trials are involved, an amazing spectrum of possible priors lead to very similar a posteriori expectations.

The weakness of the frequentist approach is that in the end you don't have any expectation, just the idea that your null hypothesis is unlikely.

  • 4
    $\begingroup$ I think a Bayesian would consider having the result depending on prior information a strength rather than a weakness. And that different folks might have different priors is also a strength and not a weakness. $\endgroup$
    – JimB
    Jul 18, 2016 at 19:20
  • 1
    $\begingroup$ Being a Bayesian myself, I agree with you. But the critiques that are most logical all center on the need for priors, and on the facts that suitably chosen priors can lead to arbitrary results and there is no natural compact space of candidate priors so there is no way to say that "these special headache priors are too unlikely to consider." $\endgroup$ Jul 18, 2016 at 20:59
  • 2
    $\begingroup$ Question: Is there an easy, analytical, and/or otherwise accepted way to measure how robust a posterior is with respect to a prior? How do I actually know if an amazing spectrum of priors will lead to similar posterior distributions? $\endgroup$
    – user541686
    Jul 19, 2016 at 7:40
  • 10
    $\begingroup$ Obligatory reference to pertinent xkcd comic. $\endgroup$ Jul 19, 2016 at 11:40
  • 2
    $\begingroup$ @Mehrdad: There is an analytical way to quantify the robsutness of some properties of the posterior distribution with respect to some parameterized family of priors. For example, if the family is "$p$ is gaussian-distributed with mean $\mu \in (0,1)$ and $\sigma \neq 0$" and the properties of the posterior we care about are the mean and variance, we can compute the derivatives of those with respect to the parmaeters describing the prior. But if you allow any function as your prior, then no, there is not an accepted or easy way to characterise the roboustness of the posterior. $\endgroup$ Jul 19, 2016 at 15:43

Is the probability of flipping heads/tails still .5 each?

If you already had the assumption that the coin is fair, then yes. Although statistically it's highly unlikely to flip heads a "very large number of times" in a row, the probability will not be changed by past outcomes. This is because each coin toss is independent from all other coin tosses.

Or has it changed in favor of tails because the probability should tend to .5 heads and .5 tails as you approach an infinite number of trials?

Already answered above but here's more detail. The law of large numbers (LLN) says that if we flip a fair coin a bunch (thousands, millions, billions, etc.) of times in a row, the outcomes should be approximately half heads and half tails. But LLN doesn't tell you anything about the probability of the next (or any) trial.

  • 10
    $\begingroup$ Thank you for posting the simple, sane answer. I believe this is what OP was asking for. The higher voted answers don't address the question of "Assuming a fair coin, does history affect probability of future event?" This one does. $\endgroup$
    – RubberDuck
    Jul 18, 2016 at 22:07
  • 4
    $\begingroup$ @Wildcard, OP said, and I even quoted it at the top of my answer, "Is the probability of flipping heads/tails still .5 each?" OP's use of the word "still" means the probability was .5 each to begin with, which is by definition a fair coin. And for clarity I said immediately after the quote, "If you already had the assumption that the coin is fair..." So while this assumption may be rather large, I feel that it was indeed implied in the question and in any case it was certainly made clear in my answer. The case of unfair coins has been left to other users whose probability skills surpass mine. $\endgroup$
    – user307169
    Jul 19, 2016 at 2:54
  • 2
    $\begingroup$ "Basic understanding of probability suggests that the probability of flipping heads is .5 and tails is .5." rather than "by assumption the coin is fair"...but this is still a well-written answer to the less interesting (to me) and simpler question, and you do state your assumption clearly at the start. So +1. :) $\endgroup$
    – Wildcard
    Jul 19, 2016 at 3:05
  • 2
    $\begingroup$ @Wildcard, I agree that there is a difference between "fair coin" and your quote of OP, but my experience has been that when students say what OP said (or things like it), they're really thinking of fairness. It's an assumption on my part and I'm not saying I'm 100% sure I interpreted correctly, just that my interpretation of OP is based on my previous teaching/tutoring experience. Thanks for the +1, and I +1 your +1! $\endgroup$
    – user307169
    Jul 19, 2016 at 12:45
  • 2
    $\begingroup$ From a purely practical standpoint, there is also the experimenter's fallacy of ignoring critical elements of the experiment. They may go to great pains to establish the fairness of the coin itself (weight, balance, symmetry, etc), but then may not be so rigorous with the flip. One can easily imagine a highly controlled flipping machine, for example, that would always flip a fair coin such that it lands as it was laid. It's not just the fairness of the coin that must be considered, but the randomness of the flipping process. Both are required to be high to realize the expected outcome. $\endgroup$
    – J...
    Jul 19, 2016 at 17:26

Probability is a mathematical model. You need to separate the modeling process from the application of the model.

We can model a coin toss by a two-element probability space {H,T} with a probability of 0.5 assigned to each element. If you sample 100 times from this probability space, and get 100 heads, then the probability of a head on the next sampling is 0.5. The probabilities are assigned in the set-up of the model and they don't change.

The question can therefore be rephrased from "Is the probability of flipping a Head 0.5?" to "Is the model appropriate to this coin?" This allows you to separate the purely mathematical part of this from the modelling process.

To answer the modelling question a Bayesian approach would make the most sense to me. You would need to quantify your confidence in you prior assumption (that a 0.5:0.5 model is appropriate) in some way. Since different people would have different levels of confidence, it is natural that different people would end up with different opinions about the appropriateness of the model. The Bayesian approach allows one to quantify what Carl Sagan meant by "Extraordinary claims require extraordinary evidence".


While we know that the proportion of heads we get from flipping a fair coin should be about half, it's worth looking at the expected amount of deviation:

If you flip a coin a million times, it's fairly rare to get exactly half a million heads: that happens with less than 0.1% probability! Even having the error be 100 or less is uncommon: that happens with about 16% probability.

In fact, if you flip a coin $n$ times, on average the the average amount of error* you get is around $\sqrt{ \frac{n}{2 \pi} }$.

So, suppose you flip a fair coin 100 times in a row and got heads every time — a surprising result saying you got 50 extra heads than average. If you then continued to flip it a million more times... by that point the average size of the error is around 1596, so that extra 50 flips isn't noticeable at all anymore.

So if you get a surprising number of heads early on, the coin has absolutely no need to be skewed towards flipping tails in order for the future statistics to behave like a fair coin usually does.

*: More precisely, this is (approximately) the expected value of $|H - \frac{n}{2}|$, where $H$ is the number of heads.


This is a great question that digs at the heart of how we use probabilities. There are several great answers, but I'd like to offer one which is more intuitive in nature.

The key to understanding the issue you are dealing with is understanding your assumptions. Let us say that you have a coin which, when flipped, lands heads 50% of the time and tails 50% of the time. This is a fair coin. You flip this coin 100 times and it lands on heads every time. It still has a 50/50 chance of landing tails on the 101th toss. That's simply how probability works.

Now let's explore your case. In your case you have a coin which has never been flipped. You then state

Basic understanding of probability suggests that the probability of flipping heads is .5 and tails is .5.

This is an assumption. You are tying the real life coin to a statistical distribution, assuming you understand how it will behave. You did not state why you feel this assumption is justified, but presumably this is because you have some experience with the physics of how objects called "coins" tend to behave when flipped. You may even have a mathematical model based on a theory of how the material in the coin is distributed.

You now flip the coin 100 times. What was the expectation for the number of times it lands on heads? Well, we don't actually know. However, we have made an assumption that this flipping process will yield 50/50 odds, so you expect 50 heads. You can also then use the Central Limit Theorem to determine the variance you expect, which will be quite narrow for 100 tosses.

Then your coin actually lands on heads 100 times. What now? Nothing in the probabilities stated a fair coin couldn't land on heads 100 times. Indeed, probability suggests that one should expect that occurrence. It would be rare, but probability would state that it should happen now and then.

You made an assumption earlier: that the coin was fair. This was justified in some method, typically by an assumption that you know how coins are flipped. Rationally, we want to think of these assumptions as sacrosanct, but intuitively you know something's going on (after all, you asked a question on Mathematics.SE about it!). So let's take a rational look at it. We made an assumption, perhaps that assumption was wrong. Perhaps the coin is magnetic and we are in a magnetic field. Perhaps the coin is weighted strangely. Perhaps its just dumb luck and our assumption was actually valid.

At this juncture, statistics and probability provide two major approaches for dealing with this rational conundrum: frequentist and Bayesian. Both of these schools have good answer with lots of points on this question. The frequentist approach is to explore whether we should reject the assumption that the coin was fair to begin with. One gathers data points (such as the 100 tosses) and then makes decisions. One may decide that the probability of 100 heads coming from a fair coin from random chance is low enough that you wish to reject the hypothesis that the coin was fair in the first place. That wording is careful. It doesn't state that the coin is fair, merely that it rejects an assumption that it was fair because the data challenges the assumption.

The Baysians take a different approach, and instead try to continuously improve their assumptions. They start with the assumption that the coin is fair. Perhaps more precisely they assume that the actual distribution is very close to that of a fair coin. They then begin flipping the coin in the test. Each time they update their assumptions about the probability distribution of the coin. As they do, they develop a hypothesis that the coin is more and more unfair as they see more and more heads.

Both approaches have their merits. There's people in each camp. However, the thing they both agree upon is that the assumption that the coin is fair is an assumption worth challenging. The frequentists try to reject the hypothesis, the Bayesian try to adapt it into a more correct hypothesis, but neither camp tries to hold themselves to the assumption that the coin is fair.

Because the world isn't fair.

  • 2
    $\begingroup$ Technically your sentence Let us say that you have a coin which, when flipped, lands heads 50% of the time and tails 50% of the time. is false. It is not possible to apply mathematical probability to physical objects without an implicit use of the law of large numbers (which is also different from your statement). So I suggest you remove that sentence and instead say that a fair coin is a purely mathematical object and we just assume it models a real coin approximately. $\endgroup$
    – user21820
    Jul 20, 2016 at 5:49
  • $\begingroup$ @user21820 It is a hypothetical object, hence the use of "Let us say." This is no stranger than "assume no friction" or "assume the cow is spherical" which are used quite often as part of the storytelling process. $\endgroup$
    – Cort Ammon
    Jul 20, 2016 at 17:29
  • 1
    $\begingroup$ That's not the point. I'm saying that no coin (not even the mathematical ideal fair coin) can come up heads exactly 50% of the time, since if you flip it once it cannot come up heads 1/2 times! You just have to change your phrasing to something like "when flipped has 50% likelihood of coming up heads" or "when flipped a large number of times comes up heads approximately 50% of the time", and it would be perfectly fine. $\endgroup$
    – user21820
    Jul 21, 2016 at 1:48

Basic understanding of probability suggests that the probability of flipping heads is .5 and tails is .5

If that is what you believe, then I have a coin here of uncertain provenance and a wager you might be interested to undertake. I will of course call heads or tails, all you are required to do is put down your $100. Suddenly you aren't so confident it's a fair coin. And that's without even tossing the coin once, I was just talking

Not knowing anything about a coin is not sufficient grounds to assume that it's a fair coin[*]. There is a sense in which we might believe that an unknown coin has a 50% chance of coming up heads, and that's if we believe that of all the unfair coins in the world, probably half of them are biased towards heads and the other half equally biased towards tails. I've no idea whether that's true or not, but if it is then we would say that a coin selected uniformly randomly from all the coins in the world has a 50% chance of coming up heads, but that the conditional probability of such a coin producing a second head after coming up heads on the first toss is more than 50%.

The issue here is that before assigning a probability you need to have a model of the process that produces the outcomes you want to assign probabilities to. Just saying "it has to be either X or Y, so let's assume the probability is 50/50" might sound vaguely plausible for coins because most coins are fair or very close to it, but not all. Without making any assumptions about where this coin came from we can't come up with a probability.

has it changed in favor of tails because the probability should tend to .5 heads and .5 tails as you approach an infinite number of trials?

Not unless we believe the coin has a "memory" of its past throws, which seems physically implausible if it's just a flattish lump of metal. Otherwise the probability we assign to another head after several heads should be either 0.5 if we have convincing grounds to believe the coin to be fair (which, if it's a real coin made by a reputable mint, we do), or some larger number if we've assigned a conditional probability that it is an unfair coin, given its past performance. The latter is pretty reasonable given that we know unfair coins exist (or, more realistically, unfair coin tosses). Actually quantifying the likelihood that we're currently encountering one, requires some assumptions about the gentleman who has just taken $100 off our hands several times in a row.

Note that as you tend to an infinite number of trials, the ratio of heads to tails of a fair coin approaches 1. However, the ratio of "the number of heads plus one million" to the number of tails, also approaches 1. That is why a fair coin doesn't need memory in order to be fair. If the first few throws are all heads then it doesn't need to somehow "make up for them" later, because as you approach infinity the first few results approach irrelevant. Every good throw of a fair coin is 50/50.

The concept in experimental science of using a p-value to reject a null hypothesis is, in effect, saying, "this coin has come up heads so many times in a row that I will no longer entertain the hypothesis that it is a fair coin".

[*] - it's effectively impossible to make a coin so biased that it has a good chance of coming up heads over and over again when thrown well. That is, barring a double-headed coin, or a "coin" with some peculiar profile that we wouldn't accept really is a coin. But if it's allowed to fall to the ground I believe non-trivial bias is possible, and if the con-artist flips it then we're in business. Mathematics lessons contain more problems about biased coins, than suggestions how to actually make one that looks anything like a normal coin, so you need to be more wary of biased coins in fake problems than in real life. But you can just expand the subject a little -- not whether the coin is fair, but whether your technique for tossing it is fair.

  • $\begingroup$ Of course "a coin" in mathematics is effectively just a label for "a two-outcome random process". For the same reason the possibility that the coin may end up on neither side is usually neglected. $\endgroup$
    – celtschk
    Jul 19, 2016 at 6:35
  • $\begingroup$ @celtschk: right, what's really happening is that "a sequence of independent random variables on the set $\{H, T\}$" is held not to be so exciting and relevant to the student's real-life experience as "a sequence of coin tosses". $\endgroup$ Jul 19, 2016 at 8:32

If a coin came up heads 1000 times in a row, I'd suspect that it was NOT a fair coin. Maybe both sides are heads, or the tails side is heavier, or there are hidden magnets involved, or some other magic trick. If I saw a coin come up heads 1000 times in a row, I'd bet that the next toss will be heads again, not tails.

But assuming that we somehow know that it really is a fair coin: One flip cannot affect the next flip because the coin has no memory. How would it "know" that it had come up heads 1000 times so now it's "due" to come up tails? Unless there's electronics hidden inside the coin recording each result and affecting future results.

Suppose you tossed a coin but didn't let me see which way it turned up. Now I toss the coin. Are the results of my toss affected by the result I didn't see? What if today I toss a coin 10 times and it comes up heads all ten times. Then I don't toss it again until tomorrow. Do yesterday's results affect today's toss? What if I don't toss it again for a week or a month? What if I never toss it again in my life, but I leave it to my son, and he leaves it to my grandson, and a hundred years from now somebody tosses it again. Will that toss be affected by the result a hundred years before?

All coins should have a 50/50 chance of coming up heads. What if you toss a coin and it comes up heads 100 times in a row, and then I toss a different coin? Is it more likely to come up tails to make up for your coin? What if someone in the next room gets heads 100 times in a row? Someone on another continent?

Suppose I have some trick to make the coin come up heads. Like, I've figured out how to toss the coin in just the right way so it flips over in the air exactly twice and comes down with the same face up that it had before the toss. Does "cheating" like this affect future fair tosses?

Etc. While you may think that intuitively, if it comes up heads a few times in a row it's now "due" to come up tails, the more you think about it the less sense this makes. There's no way for one toss to affect the next.

Lady Luck has permanent amnesia. A truly random event is, by definition, not affected by previous events.


A given process has a certain probability distribution. This distribution does not change due to the fact that you have performed that process finite many times and got a sequence of results that is very unlikely (unless you alter the process in that way).

In principle you can only get a knowledge of this probability distribution if you perform the process infinite many times, which isn't possible. Think about it like this: Any finite unlikely sequence can be evened by an infinite number of results that come after it. Compared to infinity, any finite result is negligible!

This is why an experimental model based on some experimental data (e.g. the finite sequence of results) is always given with respect to a certain confidence level. This confidence level represents a likelihood that the model is correct (e.g. can make predictions) based on that sequence of results.

Suppose your model is that your coin is a fair coin and as such obeys the binomial distribution: the number $k$ of heads within a sequence of $n$ measurments has a probability of $p_{k,n} = \binom{n}{k} \cdot 0.5^n$ to occur. Say you perform $n=100$ measurments and get $k=100$ heads. The probablity for that is $p_{100,100} \approx 7.89 \times 10^{-31}$. Pretty unlikely. However in 1 out of $1/p_{100,100} \approx 1.27 \times 10^{30}$ cases you would be correct to say that your coin is a fair coin. So in 1 out of $1.27 \times 10^{30}$ cases the next flip would have equal probability to show head or tail. In $1.27 \times 10^{30} - 1$ cases the coin would be pretty unfair though.

The problem here is, that you cannot know with certainity whether or not your coin is a fair coin. Now you can choose to standpoints: You can either say that

  1. your coin is a fair coin with a confidence level of $p_{100,100}$ or
  2. your coin is unfair with a confidence level of $1 - p_{100,100}$.

The confidence level gives the likelihood of your statement to be correct based on your performed experiment.

If you choose the second standpoint your expected probablity distribution has changed significantly from fair to unfair. The real probablity distribution is, however, unchanged. The coin will continue to be fair or unfair regardless of what you say.


There is a remote possibility that a coin could be made where the result of one throw would influence the result of the next throw. Well, just make a coin that weighs 50kg and it will probably come up the same way all the time :-)

Ignoring that possibility, the probability for that coin for throwing heads or tails doesn't change. However, each throw gives you some information about the probability that the coin is biased, and which way. I have in my drawer five coins, which will come up heads with probabilities 0, 1/4, 1/2, 3/4 and 1, respectively. You take a random coin out of my drawer. The probability of getting heads in your first throw is 1/2, but you don't know the probability for the particular coin you took. But if you had five throws and five times heads, it's highly likely that you picked the coin with 3/4 or 1 probability. If you have 95 heads in 100 throws, it is close to certain that you picked the coin with 3/4 probability and by unlikely coincidence got lots of heads.

  • $\begingroup$ 50kg... I was imagining tossing a normal-sized person and seeing if they come up heads or tails. But that exercise would probably be self-limiting and not lead to a long run of any outcome. $\endgroup$
    – user317457
    Jul 20, 2016 at 15:57

I think the question should have been put differently: Under the assumption that there is a fixed but unknown probability of heads $p$ can one predict or estimate its value with some degree of accuracy based on the results of the previous tosses(our data).

Now, the LLN says that under specific conditions when an experiment is repeated ad infinitum the empirical average of the outcomes comes very close and with great certainty to the theoretical average ie the mean value of the random outcome. In the case of the coin it means that if the trials are independent and the (unknown) probability of heads remains constant throughout the experiment then the frequency of appearances of heads approaches very closely and with great certainty the theoretical(and still unknown) probability of heads.

So based solely on the data one can say that the probability of heads is approximately the frequency of appearances of heads. If we see countless heads in a row and we accept the above assumptions we would be almost certain that the next one will be head.

Now if one is Bayesian, wants to play with $p$ and puts a prior uncertainty on that then one must calculate a quantity called posterior predictive probability ie Prob(head in the next trial| data) where data=the results of all previous tosses.

But as Bayesian estimates are a combination of empirical(data) and prior information when the size of your sample is very large the empirical part is dominant so I would say that the frequency of heads in a infinite number of tosses is again a safe estimate of the probability of heads.

  • $\begingroup$ @MarkFischler one should not put a normal prior to a proportion. The domain of the prior must agree with that of the parameter. The normal density takes values in $(-\infty,\infty)$ whereas a proportion lies between 0 and 1. If you put a normal prior to a proportion you could potentially assign positive probability to negative numbers or to numbers greater than 1. $\endgroup$
    – theoGR
    Jul 21, 2016 at 18:38

Also, to play devil's advocate, remember there is also the Gambler's Fallacy theorem.

"I will wager a bet on either red or black. There is a 1 in 2 chance of either outcome. There has been 49 red, so the next outcome must be black!"

Each incidence is a specific, discrete event that is unrelated to the other (all other factors considered, in relation to the elimination of bias, undue influence and assumption of all variables equal across each occurence of the spin of the roulette wheel).

In situations were the outcome is truly random, the assumption of balancing is erroneous and fallacious logic.

  • 2
    $\begingroup$ A Fallacy can't really be a Theorem, can it? It would be like "False Knowledge". $\endgroup$
    – user317457
    Jul 20, 2016 at 14:33
  • $\begingroup$ A fallacy is a set of mistaken beliefs. A theorem is a truth established by means of accepted truths. So whilst people who succumb to the Gambler fallacy are themselves wrong.....the theorem would kick in due to the fact that it IS a truth that people who assume that "the roulette wheel has a memory" are wrong, which itself IS a truth. $\endgroup$ Jul 20, 2016 at 15:33
  • $\begingroup$ I think that you are referring to Cognitive Bias: "a systematic pattern of deviation from norm or rationality in judgment, whereby inferences about other people and situations may be drawn in an illogical fashion." Knowledge about how people think is not itself a Theorem. Otherwise I think that would mean that everything is a Theorem... But is that a theorem? My head hurts. I guess my point was that the word 'theorem' after 'fallacy' is not needed and seems perplexing. I didn't downvote, by the way. I almost never do. "And that's a fact." $\endgroup$
    – user317457
    Jul 20, 2016 at 15:50

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .