# “confidence” level of a calculated probability

I am working on a program to solve Hangman. When you place a correct letter in the word, for example,

_ _ _ E _ S

I calculate which words this could be, and then of those words, I calculate what the next likely letter you should guess. So for in this example, there are 423 possible words, and of those, 272 words contain the letter "R". The next most likely letter is A, with 141 out of 423 words. So I can say that there is a roughly a 64% chance that the word contains the letter R.

However, as the number of possible words decreases, the "likely" letter becomes less reliable or valid. For example, if the known letters now becomes:

_ _ T E R S

There are 21 possible words. The next likely letter is "A", with 9 out of 21 words, and there is a 42% chance of it containing A. However, because the size of the possible words is so small, I don't think that 42% represents a true "probability", and my confidence would be much less for this than for the R example above. I vaguely remember a concept that corresponds to a "confidence" level for a probability, and the less samples you have, the less confidence is generated, but as you increase the number of samples, this increases.

Could anyone point me to what this concept would be?

-
When you say "there is a roughly a 64% chance that the word contains the letter R", it seems you're assuming that the word to be guessed is drawn with uniform distribution from all possible words? If so, there's nothing more to say than that there's a 42% chance of the word containing A; that number contains all the information you have about the occurrence of A, regardless of whether this arises from $9$ out of $21$ words or $900$ out of $2100$ words. – joriki Jun 29 '12 at 16:03

## 1 Answer

There is no uncertainty associated with the calculation. So the proportion you compute is not a sample estimate. If it were a sample estimate of a true proportion then you could talk about a confidence level that true proportion would fall in a confidence interval that you could generate. If you changed the problem so that instead of identifying all possible words with (as in you example) 6 letters ending in TERS, you select k words at random from the set of all such words the proportion you calculate from the example would be an estimate and you could construct a confidence interval for it at a given confidence level.

-
Okay, thanks. Both you and joriki had the same response, I didn't fully understand the concept, so I'll look into this in more detail. Thanks again – steve8918 Jul 2 '12 at 6:35