# If I am 80% sure, is there an 80% chance that I'm right? [closed]

A friend of mine said this, and I couldn't argue why I thought it was wrong.

Obviously, in some circumstances it is valid, say if you flip a coin, and I say I'm 50% sure it lands on heads.

In a discussion about whether sending a "dick pic" was illegal, he said:

I'm 99.3% sure, so there's only a 0.7% chance that I'm wrong.

Now, if we assume that it is illegal, I would agree, but isn't there a 50% chance that there is a 99.3% chance that he is wrong?

Could someone explain to me if and how this is wrong?

## closed as off-topic by Masacroso, Matthew Towers, Guy Fsone, Jack M, Parcly TaxelDec 11 '17 at 15:44

This question appears to be off-topic. The users who voted to close gave this specific reason:

• "This question is not about mathematics, within the scope defined in the help center." – Masacroso, Jack M, Parcly Taxel
If this question can be reworded to fit the rules in the help center, please edit the question.

• I'm 100% sure your friend is wrong. – aslum Dec 11 '17 at 14:17
• Once he has uttered a statement like that, it is either wrong or right, there is no probability to it. You don't know which one it is, but it's not randomly chosen in any way. – Arthur Dec 11 '17 at 14:58
• There are people who are $100\%$ sure God exists and there are people who are $100\%$ sure God does not exist. If they are both right, we have a contradiction, or we assume that both can happen simultaneously, but there are also those who are $100\%$ sure that that is impossible. The 'probability' that someone says he or she is 'sure' is not the same as the probability it will really happen. This is also more a (soft) philosofical question than a math question. – Václav Mordvinov Dec 11 '17 at 15:21

I think the main problem here is that "he is sure" is something else than "he is right". He might be 100% sure but still dead wrong, while when there is a 100% chance that he is right, then he is right.

In the sentence you highlighted, he is assuming that his being sure is equivalent to his being right. Of course, a sentence like

The chance that I am right is 99.3%, so there is only a 0.7% chance that I'm wrong.

is completely correct, but he doesn't know the chance that he is right, he just gives some random, high number to convince you.

• That's an important distinction. I knew the number was arbitrary, but I couldn't argue why the logic was wrong. – Chris Wohlert Dec 11 '17 at 11:54

A probability is a measurement about your current beliefs and confidence, not an objective statement about the outside world.

Suppose a dealer picks a playing card out of a deck in front of you and me, and hides the card from view. The dealer asks for the probability that this card is the ace of diamonds.

1. Because I didn't see the card at all, it's right for me to say that I believe there's a 1/52 chance that the card is the ace of diamonds.

2. You accidentally saw the card as it was being drawn, but only enough to see that it was a red card. It's right for you to say that you believe there's a 1/26 chance that the card is the ace of diamonds.

3. The dealer has read the card. It's right for the dealer to say that there's a 100% chance (or a 0% chance) of the card being the ace of diamonds.
4. A fourth person joins the table and for no apparent reason is suddenly struck by the idea that the card really really ought to be the ace of diamonds. The newcomer announces that the probability of the card being the ace of diamonds is 99.3%.

There is a different correct probability assignment for each person. Each person expects to be right a certain fraction of the time, but this is on the basis of different amounts of knowledge. You might even be like the newcomer, and assign a degree of confidence which is unwarranted by the data you have.

I would expect that if a person assigns the right probability corresponding to their level of knowledge (like the first three people), that person should be correct that same fraction of the time. If, however, a person assigns probabilities that don't respect the information they have (like the fourth person), there's no guarantee that expectations will agree with reality.

• "Each person has their own correct probability assignment." Um, pretty sure the 4th person's probability assignment is not correct – Kevin Dec 11 '17 at 13:56
• @Kevin The other three people feel the same way. We all have our own priors. – Sneftel Dec 11 '17 at 15:00
• @Sneftel Maybe if there were an "intuition.stackexchange.com" this answer would make sense. But since this is math.stackexchange.com, I have to downvote – Kevin Dec 11 '17 at 17:13

The way I read it your friend claims he has the following 'superpower':

Of all the gazillion times (over the course of his entire life) that he claims he is 99.3% sure of something, in 99.3% of cases it later turns out he is actually right. Similarly, in all of the bazillion cases (over the course of his life) he claimed to be 60% sure of something he turned out to be right in 60% of the cases and similar for 10%, 100%, 5% etc.

1. If one insists on making statements about levels of certainty this is an incredibly useful superpower to have or at least to strive for. It provides the statements with a very natural interpretation that makes it easy for the listener to judge its value.

2. Humans are generally very bad at estimating probabilities and so the chances that your friend actually has the above superpower are rather slim.

However your friend is not the only one making statements of this kind. The National Cancer Institute in the United States for instance has a 'tool' that, based on a number of questions about a person produces a probability that she will develop breast cancer. (Link: https://www.cancer.gov/bcrisktool/). This is an example where the makers strive to comply with what I call the 'superpower' above. I.e. the claim (on the 'about' page of the website) that the model "has been shown to provide accurate estimates of breast cancer risk" means that among women for which it predicts X% risk, a number close to X% actually develops breasts cancer within the given time interval, and that for all X.

Of course one needs criteria how close 'close to X%' should be to call the model 'accurate' and how much this closeness or lack thereof may vary depending on X. There are various answers to this question, some of which are discussed in the references in the 'about' section of the webpage.

If a statement is true with $80$% certainty, then out of every $100$ trials, the statement will be true around $80$ times. So yes, there's an $80$% chance that the statement is true in a given trial.

This question boils down to the statistical concept of calibration. A system that makes predictions (i.e. your friend) is well-calibrated if its probabilistic predictions are borne out by empirical evidence. Without knowing how well-calibrated your friend is, it is impossible to say if his estimates of probability are reliable. Certainly him stating a probability estimate doesn't make it so, we need more information about if those estimates are any good or not.

For example, if your friend makes the statement that he is 75% sure of something, and is actually correct 75% of the time of these predictions, he is well-calibrated in this range. On the other hand, if events that he predicts will happen with 30% probability actually occur 50% of the time, he tends to underestimate likelihood and is not well-calibrated.

If your friend is well-calibrated, then yes, his statements of likelihood are in fact very close to the true likelihood of an event occurring. If he is 99.7% sure of something, then he will only be wrong 3 times in 1000. However, if your friend is poorly calibrated, his estimates of likelihood don't really have any bearing on reality. He could be 99.7% sure of something and still have it occur 0% of the time.

You can test your friend's calibration with a simple online test, found at http://calibratedprobabilityassessment.org/. This tests asks a series of general knowledge questions. The goal isn't to answer them all correctly, but to correctly judge your confidence in your answers. For each question, you must select one of two options and rate your condfidence in the answer from 50-100%. A well-calibrated person will get roughly 60% of the questions right that they are 60% sure of, 80% of the questions right that they are 80% sure of, and 100% of the questions right that they are certain of.

• "General knowledge" is apparently 80% American geography. – Jacob Raihle Dec 11 '17 at 15:03
• @JacobRaihle You appear to have selected the US geography test... there are two others. – Nuclear Wang Dec 11 '17 at 17:11
• Ah, thanks. For the record, I picked "all questions", and the amount of geography questions just dwarfed the others, so international users may want to avoid that option. – Jacob Raihle Dec 11 '17 at 17:16
• @JacobRaihle But that's the beauty of calibration - you need no domain expertise whatsoever! If asked about a topic you have no knowledge about, just respond with 50% certainty. It's possible for you to guess completely randomly and still have perfect calibration. It's not a measure of how much you know, it's a measure of if you can tell whether you know or not. – Nuclear Wang Dec 11 '17 at 17:19
• Yes, it's not wrong per se, but just picking the first option and the "51-60%" bracket gets really boring. – Jacob Raihle Dec 11 '17 at 17:22

Saying that you are "sure" something is true, is not the same as it actually being true.

In cases where something is true or false, and someone makes a guess of the outcome there is a 50% chance they will be correct, and a 50% chance they will be wrong/

In this case if your friend says he is "99.3% sure" that the answer to the statement is "true". However there is nothing to say it is actually true - it is merely a hunch that he has.

The chance of him being right is still 50%.

• Nitpick: I agree with your reasoning, and the fact that sureness doesn't effect higher probability of being correct, but I would like to dispute that any statement that is either true or false has a 0.5 probability of truth. Besides the sure things, say, that 1+1=2, there are many statements that have higher or lower truth probabilities. For example, "that die, if rolled, will land on a six" has a low but nonzero probability, assuming an unknown arbitrary die, while "that die, if rolled, has 0.05 or greater probability of landing on a 6" is much more likely to be true. – timuzhti Dec 11 '17 at 15:01
• The fact that there are only two possibilities (true and false) in no way makes those possibilities equiprobable. If I buy a lottery ticket, it either is a winner or it isn't - but that doesn't mean I have a 50% chance of winning! – Nuclear Wang Dec 11 '17 at 17:15