Suppose we have an experiment which results in "success" with probability $p$ and in "failure" with probability $1-p$. The distribution of corresponding random value is assumed to be Bernoulli.

It turns out that after a hundred experiments there were no "successes". How can one assess $p$ with some confidence level (for example, $0.95$)?

My attempt. The probability that after a hundred experiments there will be no "successes" is $\left(1-p\right)^{100}$. So one can compare this probability with the given confidence level. That means that $$(1-p)^{100}\geq0.95$$ That gives us that $$0\leq p\leq 1-\sqrt[100]{0.95}\approx0.0005128$$ Is that approach correct? If no, how can one tackle that problem?


I don't think so. If you're trying to establish the range of $p$ over which the given result is not significant at the $0.05$ level, then you want

$$ (1-p)^{100} \geq 0.05 $$

and then $0 \leq p \leq 0.0295$ approximately. What you have there instead finds the range of $p$ over which the probability of obtaining a result that extreme is at least $0.95$, which I don't think is what you meant.

Alternatively, one can take a Bayesian approach, and use some appropriate prior (Beta distribution with $\alpha = \beta = 1$, which is uniform, will work), and update it according to the $100$ consecutive failures. The result (if you start with a uniform prior) is a Beta distribution with $\alpha = 1, \beta = 101$, whose PDF looks like this:

enter image description here

  • $\begingroup$ I would argue that you should use $(1-p)^{100} \geq 0.025$ if you want to use a "two-sided" 95% interval where each tail contains a maximum mass of 0.025. This gives $[0, 0.0362167]$ as the confidence interval. $\endgroup$ – PhiNotPi Apr 23 at 17:50
  • $\begingroup$ @PhiNotPi: That would depend on the circumstances, but sure, I could see situations where that'd be what you want. $\endgroup$ – Brian Tung Apr 23 at 18:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.