Suppose I have a coin and I suspect it is unfair. I've flipped it ten times and gotten heads without fail. As this question explains, the most likely bias* of the coin is 1. That is, it will always land heads-up. However, it is also possible that it is a fair coin, and we have just witnessed an event which has probability $\frac12^{10} \approx 0.001$. I would like to know a formula to be able to make claims such as "We can say with 95% confidence that the bias of the coin is greater than or equal to 0.98," ideally generalizable to any sample of $h$ heads in $n$ trials.

*I define the bias $b$ of a coin as the actual probability of heads such that as the number of trials grows arbitrarily large, $b$ approaches $\frac{\text{heads}}{\text{total}}$

  • $\begingroup$ The statement "We can say with 95% confidence that the bias of the coin is greater than or equal to 0.98" does not make sense. $\endgroup$ – kludg Mar 25 '19 at 15:55
  • $\begingroup$ @kludg I admit I haven't done statistics in a while. I remember studying confidence intervals and being able to use a sample of observations to say something like "We have 95% confidence that $\mu$ is between 5.6 and 7.8". For example, we can say with 100% confidence that $b > 0$. I'm imagining a cumulative graph whose area would be 1, where the x-axis denotes $b$ and the y-axis denotes the probability that $b$ equals that x-value. Perhaps my question is simply not well-formed enough to be answered meaningfully. If so, I apologize. $\endgroup$ – Jon McClung Mar 25 '19 at 16:08
  • $\begingroup$ You can have 95% confidence interval but you can't be 95% confident in a particular observation. This is old story of confidence interval interpretation. IMO what you are asking is better be formulated in terms of hypothesis testing, say you have null hypothesis "coin is fair", alternative hypothesis "coin is unfair" or even "coin bias is > 0.98" and want to reject null hypothesis by alternative hypothesis in a statistical test of some significance level, say 5%, etc. But maybe I don't understand you. $\endgroup$ – kludg Mar 25 '19 at 17:57
  • $\begingroup$ @kludg Yes, that interpretation seems more in line with what I'm looking for. $\endgroup$ – Jon McClung Mar 25 '19 at 18:19

I'd recommend to read the following Binomial Proportion Confidence Interval. If the sample size is large enough and the proportion is not close to $0$ or $1$ you can use the normal distribution approximation. As you don't have the variance of the population, you can use a $t$ distribution.

However if you want an exact answer for the interval, you can do the following (I'm quite sure it's not theoretically accurate and you need additional hypothesis) but for practical reasons you can always normalize the likelihood function and use it as a pdf for the parameter.

In this case $L(p|x)={n \choose h} p^{h} (1-p)^{n-h}$.

Let's say you maximize this likelihood function and obtain parameter estimation $p^{*}$ (in this case $p^{*}= \frac{h}{n}$). So build your interval limits $[p^{*} - l, p^{*} + l]$ the following way, if you want confidence $\alpha$, then find $l$ given that:

$$\alpha = \frac{1}{\int_{-\infty}^{\infty} \mathbb{1}_{p \in [0,1]} {n \choose h} p^{h} (1-p)^{n-h} dp} \int^{p^{*}+l}_{p^{*}-l} \mathbb{1}_{p \in [0,1]} {n \choose h} p^{h} (1-p)^{n-h} dp$$

It is important to note that this answer assumes a symmetric interval of confidence around $p^{*}$ (obviously within the domain for parameter $p$ which will be limited by the indicator function in the integral).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.