Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

We are very rusty on our math, probability, statistics, permutations and combinations and need some help figuring out how to calculate some relative "confidence level" based on the number of people who give the same answer to a question. This is somewhat related to Wisdom of Crowds where if we ask a group of people the same question and many choose the same answer it is likely that answer is correct.

Answer to a question is either "a" or "b". If we ask 1 person we would have 50% confidence that is the right answer. If we asked two people and they both said the same answer we would be 75% confident it is right I think??

If we keep extending this:

Possible Answers = 4, we ask 2 people and they give the same answer ... it should be a higher confidence level but what? (87.5%?)

The more possible answers for a question and the more people who give the same answer should lead to a higher confidence level - but what is the formula or algorithm to calculate it?

share|cite|improve this question
Related: – Charles Apr 28 '11 at 13:59

No, the 50% is already totally arbitrary. If you do not know anything about a probability distribution it does not imply equidistribution.

If you actually assume that people give random answers then there is no information at all in the fact that three people give the same answer.

share|cite|improve this answer
What if we assume that everyone has a probability $p > .5$ of being right? – Joshua Shane Liberman Apr 28 '11 at 13:59
Oh, I agree that a meaningful question can be formulated in this context, but building the model comes first and after watching friends learn statistics for their economics degrees, I think that it is dangerous to give people formulas before they can understand their models. They will plug in stuff. The Wisdom of Crowds is based on the model that there is the same chance to guess too high than to guess too low which may be reasonable for guessing the sweets in a jar. – Phira Apr 28 '11 at 15:09

Refer to the computation of a binomial confidence interval.

Your "P" in the formula would be .5, which is the probability of being correct, by chance, given choices "a" or "b". "n" would be the number of people surveyed and "z" is the z-score. Once you obtain this confidence interval, subtract this result from 1.

The normal approximation formula is given as:

$$\hat{p}\pm z_{1-\alpha/2}\sqrt{\frac{\hat{p}(1-\hat{p})}{n}}$$

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.