101
$\begingroup$

Consider a two-sided coin. If I flip it $1000$ times and it lands heads up for each flip, what is the probability that the coin is unfair, and how do we quantify that if it is unfair?

Furthermore, would it still be considered unfair for $50$ straight heads? $20$? $7$?

$\endgroup$
  • 26
    $\begingroup$ Here's a wikipedia article devoted to exactly this question. $\endgroup$ – Omnomnomnom Jul 2 '15 at 1:57
  • 16
    $\begingroup$ If you are on a ship sailing from Denmark to England, the odds may differ. $\endgroup$ – marty cohen Jul 2 '15 at 3:32
  • 10
    $\begingroup$ And if you continued the experiment and flipped it another 1000 times and it came up tails every time then we would have to judge it fair. $\endgroup$ – Aran Mulholland Jul 2 '15 at 7:25
  • 5
    $\begingroup$ Since "fair" and "unfair" are qualitative values, let me give a qualitative answer: For 1,000 and 50 straight heads --> I would bet my life that the coin is "unfair". For 20 straight heads --> I would not bet my life that the coin is "unfair", though it sure seems to be. For 7 straight heads --> I would consider the coin "fair" though I wouldn't wager against it without further data. $\endgroup$ – user23715 Jul 2 '15 at 15:59
  • 7
    $\begingroup$ Bear in mind, If you happen to flip 1000 times and then it occurs to you that it may be unfair, it's not exactly fair to use that data in your test $\endgroup$ – Nathan Cooper Jul 3 '15 at 8:04
103
$\begingroup$

First of all, you must understand that there is no such thing as a perfectly fair coin, because there is nothing in the real world that conforms perfectly to some theoretical model. So a useful definition of "fair coin" is one, that for practical purposes behaves like fair. In other words, no human flipping it for even a very long time, would be able to tell the difference. That means, one can assume, that the probability of heads or tails on that coin, is $1/2$.

Whether your particular coin is fair (according to above definition) or not, cannot be assigned a "probability". Instead, statistical methods must be used.

Here, you make a so called "null-hypothesis": "the coin is fair". You then proceed to calculate the probability of the event you observed (to be precise: the event, or something at least as "strange"), assuming the null-hypothesis were true. In your case, the probability of your event, 1000 heads, or something at least as strange, is $2\times1/2^{1000}$ (that is because you also count 1000 tails).

Now, with statistics, you can never say anything for sure. You need to define, what you consider your "confidence level". It's like saying in court "beyond a reasonable doubt". Let's say you are willing to assume confidence level of 0.999 . That means, if something that had supposedly less than 0.001 chance of happening, actually happened, then you are going to say, "I am confident enough that my assumptions must be wrong".

In your case, if you assume the confidence level of 0.999, and you have 1000 heads in 1000 throws, then you can say, "the assumption of the null hypothesis must be wrong, and the coin must be unfair". Same with 50 heads in 50 throws, or 20 heads in 20 throws. But not with 7, not at this confidence level. With 7 heads (or tails), the probability is $2 \times 1/2 ^ {7}$ , which is more than 0.001.

But if you assume confidence level at 95% (which is commonly done in less strict disciplines of science), then even 7 heads means "unfair".

Notice that you can never actually "prove" the null hypothesis. You can only reject it, based on what you observe is happening, and your "standard of confidence". This is in fact what most scientists do - they reject hypotheses based on evidence and the accepted standards of confidence.

If your events do not disprove your hypothesis, that does not necessarily mean, it must be true! It just means, it withstood the scrutiny so far. You can also say "the results are consistent with the hypothesis being true" (scientists frequently use this phrase). If a hypothesis is standing for a long time without anybody being able to produce results that disprove it, it becomes generally accepted. However, sometimes even after hundreds of years, some new results might come up which disprove it. Such was the case of General Relativity "disproving" Newton's classical theory.

$\endgroup$
  • 2
    $\begingroup$ This seems interesting but I think there may exist some better approach than this, maybe (Im not sure). Unfortunately I dont have by now the knowledge to make an answer. To me the uses of law of larger numbers is fundamental here or some analysis at the level of variance more than mean. $\endgroup$ – Masacroso Jul 2 '15 at 2:39
  • 7
    $\begingroup$ @Masacroso your intuition is correct. If we had, for example, 550 heads for 1000 throws, you would want to calculate not the probability that exactly 550 heads happen (which is easy but incorrect thing to do), but the probability that 550 heads or_more happened. This is not so simple and advanced techniques (of the kind you are mentioning) are needed. But in the exact question that was asked, it is OK to calculate it like I did. $\endgroup$ – Mark Galeck Jul 2 '15 at 3:04
  • 5
    $\begingroup$ How would your confidence in rejecting the null hypothesis (the coin is fair) have been different if the results had been HTHTHTHTHTH..., perfect alternation between heads and tails? Or if it had been HTTHHTTHHTTHHTTHH... or indeed any specific sequence of outcomes? $\endgroup$ – Marc van Leeuwen Jul 2 '15 at 6:20
  • 2
    $\begingroup$ @MarcvanLeeuwen Marc, note what I said in my comment - you actually don't calculate the probability of any given event, because most of the time , that would be very close to 0. Instead, you correctly choose the "tail" of the distribution, for that particular null hypothesis. For this null hypothesis, the tail is "the number of heads in the event, or more". So for perfect alternation event you describe, although it is a very "strange" event, but it would not disprove this particular null hypothesis. Any event, would disprove it, only if there were far more one side than the other. $\endgroup$ – Mark Galeck Jul 2 '15 at 6:47
  • 5
    $\begingroup$ @MarcvanLeeuwen In both those cases, you could then use "I expect each consecutive pair of flips (HH, HT, TH, TT) to occur with equal probability" to be the null hypothesis, and calculate against your confidence interval similarly. $\endgroup$ – Phylogenesis Jul 2 '15 at 12:29
43
$\begingroup$

If you take a coin you have modified so that it always lands in heads and you get $1000$ heads then the probability of it being unfair is $100\%$.

If you take a coin you have crafted yourself and carefully made sure that it is a fair coin and then you get $1000$ heads then the probability of it being unfair is $0\%$.

Next, you fill a box with coins of both types, then take a random coin.

  • $NF$ : fair coins in the box.
  • $NU$ : unfair coins in the box
  • $P(U)$ : probability of having taken an unfair coin $$P(U) = \frac{NU}{NF + NU}$$

  • $P(F)$ : probability of having taken a fair coin $$ P(F) = \frac{NF}{NF + NU} = 1 - P(U) $$

  • $P(H \mid{U})$ : Probability of having 1000 heads conditioned to having take an unfair coin $$P(H\mid{U}) = 1 $$
  • $P(H\mid{F})$ : Probability of having 1000 heads conditioned to having taken a fair coin $$P(H\mid{F}) = \left( \tfrac{1}{2} \right)^{1000}$$
  • $P(H)$ : Probability of having 1000 heads

\begin{align} P(H) &= P(U \cap H) + P(F \cap H)\\ &= P(H \mid{U})P(U) + P(H \mid{F})P(F)\\ &= P(U) + P(H \mid{F})P(F) \end{align}

By applying Bayes theorem :

$P(U \mid{H})$ : probability of the coin being unfair conditioned to getting 1000 heads $$P(U\mid{H}) = \frac{P(H \mid{U})P(U)}{P(H)} = \frac{P(U)}{P(U) + P(H\mid{F})P(F)}$$

And that is your answer.


In example

If $P(U)=1/(6 \cdot 10^{27})$ ($1$ out of every $6 \cdot 10^{27}$ coins are unfair) and you get 1000 heads then the probability of the coin being unfair is \begin{align} \mathbf{99}.&999999999999999999999999999999999999999999999999999999999999999999999\\ &999999999999999999999999999999999999999999999999999999999999999999999\\ &999999999999999999999999999999999999999999999999999999999999999999999\\ &999999999999999999999999999999999999999999999999999999999999999944\% \end{align}

Very small coins like the USA cent have a weight of $2.5g$. We can safely assume that there are no coins with a weight less than 1 gram.

Earth has a weight of less than $6 \cdot 10^{27}$ grams. Thus we know that there are less than $6 \cdot 10^{27}$ coins. We know that there is at least one unfair coin ( I have seen coins with two heads and zero tails) thus we know that $P(U) \ge 1/(6 \cdot 10^{27})$.

And thus we can conclude that if you get 1000 heads then the probability of the coin being unfair is at least \begin{align} \mathbf{99}.&999999999999999999999999999999999999999999999999999999999999999999999\\ &999999999999999999999999999999999999999999999999999999999999999999999\\ &999999999999999999999999999999999999999999999999999999999999999999999\\ &999999999999999999999999999999999999999999999999999999999999999944\% \end{align}

This analysis is only valid if you take a random coin and only if coins are either $100\%$ fair or $100\%$ unfair. It is still a good indication that yes, with $1000$ heads you can be certain beyond any reasonable doubt that the coin is unfair.

$\endgroup$
  • 4
    $\begingroup$ Very good for including actual data. Bayesian works best in presence of data. $\endgroup$ – PyRulez Jul 2 '15 at 15:34
  • 2
    $\begingroup$ So your saying it probably wasn't a fluke then? $\endgroup$ – PyRulez Jul 2 '15 at 15:36
  • 7
    $\begingroup$ @PyRulez More than that. This great answer managed to show that 1) the question omitted a number of key parameters (like what is the coin population I'm looking at) without which such a question normally makes no sense, and that 2) the remaining parameters were so absurdly extreme that in any otherwise "normal" circumstances inside the Solar system, this couldn't have been a fluke. $\endgroup$ – Jirka Hanika Jul 3 '15 at 12:18
  • 8
    $\begingroup$ @PyRulez The real fluke would be that after the first 500 in a row, the flipper didn't go "Hmm... better do another 500 just to be sure..." $\endgroup$ – corsiKa Jul 3 '15 at 16:58
  • 3
    $\begingroup$ @corsiKa Well, you can never be sure. $\endgroup$ – PyRulez Jul 3 '15 at 17:21
21
$\begingroup$

In order to assign a probability to this event you need to start with a prior probability which can then be updated based on the data. Probabilities can't really be derived from experience alone, you need to start with some sort of "inductive bias" in order to draw conclusions from evidence.

The heuristic approach of hypothesis testing gives a framework for making decisions in these situations, but it doesn't pretend to assign probabilities to those hypotheses.

$\endgroup$
8
$\begingroup$

In my comment, posted this link to a more thorough treatment of the general question being asked. However, I will directly answer the question using the attained result on posterior density functions, as discussed in the linked article.

In particular, we note that if $r$ is the actual probability that the coin will land on "heads" rather than "tails", then if we got $h$ heads and $t$ tails, the distribution of $r$ is described by the probability density function $$ f(r\mid T=t; H=h) = \frac{(h+t+1)!}{h! \, t!} r^h(1-r)^t $$ Now, let's say that the coin is "fair" if $r$ (which should be exactly $1/2$) falls between $0.45$ and $0.55$. Then the probability that our coin is fair, given $h$ heads and $t$ tails, is given by $$ P = \int_{.45}^{.55}f(r\mid T=t; H=h)\,dr = \frac{(h+t+1)!}{h! \, t!} \int_{0.45}^{0.55} r^h(1-r)^t dr $$ Suppose we get $0$ tails and $h$ heads. Then this probability comes out to $$ P = \int_{.45}^{.55}f(r\mid T=0; H=h)\,dr = (h+1) \int_{0.45}^{0.55} r^h dr = \\ \left. r^{h+1} \right|_{0.45}^{0.55} = (0.55)^{h+1} - (0.45)^{h+1} $$ As you can see, the more consecutive heads we get (assuming this is all the information we have), the less likely it becomes that the coin is fair. For $20$ straight heads, we compute $$ P = (0.55)^{21} - (0.45)^{21} = 3.48 \times 10^{-6} $$ Which is to say that it is extremely unlikely that the coin is fair. The probability that the coin is unfair is, correspondingly, very high.

$\endgroup$
  • 1
    $\begingroup$ Mind that in this case, our prior distribution is uniform. So, the probability that a coin is fair (given that we have not yet flipped it), by this consideration, is $1/10$. $\endgroup$ – Omnomnomnom Jul 2 '15 at 4:44
  • 3
    $\begingroup$ Using a uniform prior IMO makes this computation useless. You are assuming that there are as many coins that give heads say with fixed probability $p$ with $0< p<0.10$ as there are fair coins with your definition ($0.45<p<0.55$); it reflects an extremely strong prior suspicion against fairness of the coin. If your prior were based on an experimental estimation of $p$ among actual coins, reflecting the fact that actual coins are very likely to be fair according to you definition, your conclusion might be "likely fair" (depending on just how small the a piori likelyhood of an unfair coin is). $\endgroup$ – Marc van Leeuwen Jul 2 '15 at 9:02
  • 2
    $\begingroup$ @MarcvanLeeuwen I would disagree about the "uselessness" of the approach, but yes: that assumption may be excessively biased towards an unfair coin. However, I think that changing the prior distribution only affects the probability by at most some constant factor, so that the rate at which the probability of unfairness increases is accurate. $\endgroup$ – Omnomnomnom Jul 2 '15 at 10:05
6
$\begingroup$

For "almost all" priors, the answer is 100%: you have to specify an interval of "fairness" in order to get a nonzero value. Why? Because the bias $p$ can be any real number between $0$ and $1$, so the probability that it is a very specific real number is $1/\infty = 0$; hence the probability that it is $1/2$ is 0.

However, what I can tell you is that after your tosses, the heads probability is $$\frac{h + 1}{t + h + 1 + 1} = \frac{1000 + 1}{0 + 1000 + 1 + 1} = \frac{1001}{1002}$$ ...assuming you initially believed the coin was fair (see @leonbloy's comment under the question).

More generally, if the heads probability $p$ has prior distribution $\Pr(p)$ then the answer is:

$$\frac{\int_0^1 p\,p^{1000}(1-p)^{0}\Pr(p) \,dp}{\int_0^1 \phantom{p\,} p^{1000}(1-p)^{0}\Pr(p) \,dp}$$

Notice that for the uniform prior $\Pr(p) = 1$, this degenerates to what I gave above.

The derivation is longer and likely to be much more complicated than you might expect, so I'll omit it to save typing... if you really want to see it, let me know.
(Basically, you need to apply Bayes's rule and simplify.)

$\endgroup$
  • $\begingroup$ Actually, an uniform prior means that you made no assumption at all about the fairness of the coin (you still get $p(Head)=0.5$ from ignorance: Since you have no bias, both results are equally likely). A prior 100% assumption that the coin was fair would mean $Pr(p)=\delta(p-0.5)$, and with that you'd find that the coin is fair with probability 1 no matter what you toss. Basically it's the "unshakeable belief" prior. In reality, you'd probably use a prior that's peaked around the fair coin (because most coins are approximately fair, but you wouldn't exclude the probability of it being biased). $\endgroup$ – celtschk Jul 5 '15 at 11:43
2
$\begingroup$

You asked

if this specific string of flips happens, what is the probability the coin is unfair?

And it seems to me that most answer here are answering the question

if the coin was fair, what is the probability of this specific string of flips happening?

If I try to literally answer your question, I get stuck unless we make additional assumptions. For your question, the sample space would have to be something like all instances ever of flipping a coin 1000 times. Not one specific coin mind you, but all instances ever, anywhere, of flipping one coin 1000 times. The next instance of 1000 flips can use a different coin. Some elements of that sample space would have 1000 heads. And some elements of that sample space would have involved a fair coin.

But we can't do any calculating until we understand, among coins, what is the distribution of fairness? $p$ for a coin can range form $0$ to $1$, with only exactly $0.5$ being fair. If you assumed a specific distribution for the fairness of all coins, then we'd have the necessary information to calculate an answer for your literal question.

In a world where every coin is fair, then even with these 1000 heads, there is a 0% chance that your coin is unfair. In a world (more like the real world) where no coin is truly fair, then there is automatically a 100% chance that your coin is unfair. If you'd like to model with something in between, say 50% of coins are truly fair, and the other 50% is distributed some way among $p\in[0,0.5)\cup(0.5,1]$, then based on that information we could calculate a more interesting answer.

$\endgroup$
1
$\begingroup$

This is an issue not so much about mathematics as about the real-world background to the coin-tossing experiment. Convincing fake coins are fairly common: for example, about 5% of the £1 coins circulating in the UK are counterfeit*. It would be just as easy to forge a double-headed (or double-tailed) coin as a normal one. And probably (!) forgers have made a few, either as a joke or to cheat people. On this basis, your coin (by a Bayesian analysis) is almost certainly double-headed.

Well, the next step is to inspect the coin to verify that it has a head on only one side. Then you have to ensure that this is actually the coin that is thrown each time; that the way it is thrown is fair; that the people checking the coin and recording the outcomes are honest and scrupulous; and so on. The more boxes are ticked, the less plausible is the 1000-head outcome. But this outcome would be near impossible even for a coin that is osmium--iridium on the tail side and magnesium on the head side. So doubts would still arise about the checking and recording procedure. For example, is a conjuror or skilled hoaxer involved at some stage?

The discussion of this experiment can obviously go hyperbolic. To cut things short, if the conditions of the experiment are guaranteed, in some absolute cosmic sense, to be fair, and only the balance of the coin is in question, then supposing the 1000-head outcome would be close to postulating a miracle; and that does not usually admit rational discussion.

As the supposed number of heads reduces, so the above argument weakens and needs to become more hedged, more complex, more numerical, and more uncertain. At some point, the only supportable response is to shrug one's shoulders and say "who knows?".

*2018 update: The £1 coin mentioned has now been replaced by a coin with new complex features to make forgery much harder.

$\endgroup$
0
$\begingroup$

The Bayesian approach others are taking requires guessing at a prior distribution of the coin's probability of heads, introducing another assumption into the question. We can simply take another approach, which is to ask "if the coin was perfectly fair, what are the chances I'd observe a string of heads of length $k$ or longer?"

It is well known that this follows the binomial distribution, which says that the probability of getting exactly $k$ heads in $n$ independent flips, each with probability $p$ is given by

$$ f(k;n,p) = \binom{n}{k}p^k(1-p)^{n-k} $$

If I want to know the probability of length $k$ or longer, in general, I have to sum up the probability for length $k$, $k+1$, $k+2$...up to $n$ flips. However, in this case, $k=n$ because we are calculating what happens when all the flips are heads, so there is only one term in the sum. If the coin were perfectly fair, then $p=1/2$. All this gives $$ f(k;k,1/2) = \frac{1}{2^k} $$

For $k=1000$, this gives a probability less than $1$ in $10^{301}$. For $k=50$ the probability is less than $1$ in a quadrillion. For $k=20$ the probability is less than $1$ in a million. For $k=7$ the probability is $1$ in $128$. It's up to you to decide if these outcomes are unlikely enough to warrant the conclusion of unfairness; that number is the probability that you got a string of heads by chance from a truly fair coin. If that probability seems unreasonable to you, you can instead conclude that the coin is not truly fair.

$\endgroup$
  • 6
    $\begingroup$ Suppose we get the combination $THTH...$ of length $k=50$. By your logic, the probability of this outcome is $1$ in a quadrillion, so it's practically impossible that it happened by chance. So, we might conclude that we have a "magic coin". But then again, the probability of any single sequence of outcomes is about 1 in a quadrillion. That's low enough to assume that the outcome that occurred couldn't have happened by chance. The conclusion we must reach, then, is that if you flip a coin 50 times, it becomes magic. $\endgroup$ – Omnomnomnom Jul 2 '15 at 15:25
  • $\begingroup$ @Omnomnomnom yes, raj245 answer is basically the same as mine, but without one very very important detail - that you should take the probability of the "tails" of the distribution, rather than, a particular event, which is likely to have a very low probability by itself. $\endgroup$ – Mark Galeck Jul 2 '15 at 16:24
  • $\begingroup$ @Omnomnomnom the string of heads and tails you propose is simply $k=25$ heads in $n=50$ flips; the probability assigned to this event is about $11$ percent, so I wouldn't say this coin is magical; this outcome is not exceedingly rare. Remember, the binomial distribution does not give the probability of one particular string of outcomes, but simply of the total counts of a string of outcomes. It just happens in the case of all heads, there is only one way to choose how to assign $50$ heads into $50$ slots. $\endgroup$ – rajb245 Jul 6 '15 at 19:33
  • $\begingroup$ @raj245 no not at all - the perfectly alternating string of results does not have 11% chance. It has the same chance as "all heads" or "all tails". $\endgroup$ – Mark Galeck Jul 8 '15 at 5:55
  • $\begingroup$ I don't disagree that any particular sequence is very rare, only that getting 25 out of 50 heads, in any order, is not that rare. If a coin was observed to give a set of flips that was THTHTH..., I would use the binomial distribution and get about 11%, and conclude that the this is not unlikely enough to discard the hypothesis that the coin is fair. As you said, we both gave answers along the same lines...it seems that we should agree here. In general, I'd sum over events more extreme (the distribution "tails" you talk about, $\sum_k f(k;n,p)$), but that sum only has one term in this case. $\endgroup$ – rajb245 Jul 8 '15 at 14:36
-5
$\begingroup$

Given the chance for a head-up is 0.5:

$(1/2)^{1000} = 9.332636 \times 10^{-302}$

50 times?

$(1/2)^ {50} = 8.8817842 \times 10^{-16}$

Depends on your standard of unfairness, I would say it is pretty unfair for cases 50 and 1000 of straight heads.

$\endgroup$
  • 6
    $\begingroup$ For the case of 1000 consecutive heads, do you mean that there is a 1 - ((1/2)^1000) percent chance that the coin is biased? What prevents us from saying that there would be a 1 - ((1/2)^5) percent chance that a coin is biased if it lands heads up five straight times? Where do we draw the line? $\endgroup$ – Adam Freymiller Jul 2 '15 at 2:05
  • 3
    $\begingroup$ You are just plain confusing the probability that $n$ successive outcomes will be heads given a fair coin with the likelyhood that a coin is fair given $n$ successive outcomes "heads". The link between these two problems (only the first of which is well formulated probability question) is complicated, and at the heart of the foundations of statistics; in fact the answer to the first problem in itself says almost nothing about the second problem. Such simplisitc answers are unhelpful. $\endgroup$ – Marc van Leeuwen Jul 2 '15 at 11:18

protected by Community Jul 3 '15 at 12:35

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.