I am given the task to check if a coin is biased to land on heads. The bias must exceed a certain threshold i.e. $p > 0.5 + \epsilon$ for some given $\epsilon$. I would like to know how the number of flips affects the certainty of the conclusion that the coin is biased according to this definition.

Concretely, if I am allowed to flip the coin $n$ times, what is the probability of a false positive (coin is not biased but I claim that it is) and a false negative (coin is biased but I fail to spot it) as a function of $n$ and $\epsilon$?

  • 1
    $\begingroup$ Please take a look at the Binomial Test. $\endgroup$ – denklo Jan 28 at 11:26
  • $\begingroup$ The question is wrong. It is no more answerable than "How long is a piece of string?". To make a sensible question, more information is needed. If we know the prior probability that the coin is biased, then we can calculate a posterior probability of it being so from the result of any given number of flips. $\endgroup$ – John Bentin Jan 28 at 12:18
  • $\begingroup$ @JohnBentin, I took a look at the binomial test. Isn't it correct to have a null hypothesis that the coin is biased such that $p>0.5 +\epsilon$ and the alternative hypothesis that it is biased but with $p\leq 0.5 +\epsilon$? I wish to accept/reject the null hypothesis. Is this a well defined question? $\endgroup$ – user1936752 Jan 28 at 13:43
  • $\begingroup$ The question didn't make clear that the set-up was to test the null hypothesis $p>0.5+\epsilon$ versus the opposite. For any $p\in[0\,\Bbb,\, 1]$, you can calculate the probability of the outcome being as extreme as, or more extreme than, what is observed given that the probability of a head is $p$. But this looks like a pretty meaningless exercise. Even this useless calculation becomes impossible if we weaken the assumption from fixing P(head) to merely P(head) > $0.5+\epsilon$ without specifying a distribution for P(head) in $[0.5+\epsilon\,\Bbb, \,1]$. $\endgroup$ – John Bentin Jan 28 at 19:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.