Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

In his book The Signal and the Noise, Nate Silver presents this example application of Bayes's Theorem on pp. 247-248:

Consider a somber example: the September 11 attacks. Most of us would have assigned almost no probability to terrorists crashing planes into buildings in Manhattan when we woke up that morning. But we recognized that a terror attack was an obvious possibility once the first plane hit the World Trade Center. And we had no doubt we were being attacked once the second tower was hit. Bayes's theorem can replicate this result.

You can view the complete example in Amazon.com's previw, and I've made the two pages available here.

Silver assumes the prior probability of a terrorist plane attack to be 1 in 20,000. After the first plane crash, using Bayes's Theorem he updates that to 38%. And after the second plane crash, he comes up with a 99.99% probability. However, I think he may be mistaken. I'll provide the details below.

To be precise, let us define the following three events:

  • $PC$ = Plane Crash: At least one plane crashes into a Manhattan skyscraper on a given day.
  • $TPA$ = Terrorist Plane Attack: At least one plane is intentionally crashed into a Manhattan skyscraper on a given day.
  • $APC$ = Accidental Plane Crash: At least one plane is accidentally crashed into a Manhattan skyscraper on a given day.

We assume all plane crashes into buildings are either terrorist plane attacks or accidental (i.e. $PC = TPA \cup APC$). Using historical data, Silver estimates the prior probability of an accidental plane crash to be 1 in 12,500. In summary: $$P(TPA) = \frac{1}{20000},$$$$P(APC) = \frac{1}{12500}.$$

Furthermore, Silver assumes $P(APC) = P(PC|\overline{TPA})$ (which is true if $APC$ and $TPA$ are independent events).

Applying Bayes's Theorem, he comes up with $$\begin{align}P(TPA|PC) &= \frac{P(PC|TPA) \times P(TPA)}{P(PC|TPA) \times P(TPA) + P(PC|\overline{TPA})(1-P(TPA))} \\ &= \frac{1 \times \frac{1}{20000}}{1 \times \frac{1}{20000} + \frac{1}{12500} \times (1 - \frac{1}{20000})} = 0.385\end{align}$$

Silver continues:

The idea behind Bayes's theorem, however, is not that we update our probability estimates just once. Instead, we do so continuously as new evidence presents itself to us. Thus our posterior probability of a terror attack after the first plane hit, 38 percent, becomes our prior probability before the second one did. And if you go through the calculation again, to reflect the second plane hitting the World Trade Center, the probability that we were under attack becomes a near-certainty -- 99.99 percent.

That is (this is Silver's calculation): $$P(TPA|PC) = \frac{1 \times 0.385}{1 \times 0.385 + \frac{1}{12500}(1-0.385)} = 99.99 \%$$

"Cool!" I thought, until I thought a bit more. The problem is that you can apply the same logic to calculate the conditional probability of an accidental crash, too. I'll spare you the math, but I come up with $P(APC|PC) = 0.615$ after the first crash, and $P(APC|PC) = 99.997\%$ after the second.

So we can be almost certain the second plane crash is a terrorist attack, and we can be even more certain that it's accidental?

I think the problem is that when Silver applies Bayes's Theorem after the second crash, he uses the updated probability of a terrorist plane attack as his prior, but fails to update the prior probability of an accidental plane crash (which should become 0.615). After the second crash, then, the correct formula is $$P(TPA|PC) = \frac{1 \times 0.385}{1 \times 0.385 + 0.615(1-0.385)} = 0.504$$

Similarly, the probability that we're observing an accidental crash given that there have been two crashes is $$P(APC|PC) = \frac{1 \times 0.615}{1 \times 0.615 + 0.385(1-0.615)} = 0.806$$

Question 1: Am I correct that Nate Silver is doing it wrong?

Question 2: Am I doing it right?

share|improve this question
    
It seems like this reasoning could be applied to anything - if you flip your null hypothesis you're likely to get a different result. –  Cocopuffs Jan 4 '13 at 4:14
    
You computed the conditional probability of something being true on some observations, then you computed the conditional probability that it's false on the same observations, and the two probabilities don't add up to $1$. So that's bad. –  Qiaochu Yuan Jan 4 '13 at 4:21
    
When you apply Bayes' theorem the second time, you need to be clearer about what your hypotheses are. A priori there are four hypotheses depending on which of the two planes was an accidental crash vs. a terrorist crash, and if one of the planes was a terrorist crash it's much more likely that the second one is too, so you need to take that into account. –  Qiaochu Yuan Jan 4 '13 at 4:23
    
Qiaochu: Regarding the first comment, APC and TPA aren't disjoint (mutually exclusive) when there have been 2 or more crashes, so the probabilities need not add up to 1. There is merit to your second comment. But the problem is that after the first plane crash, we don't know whether it was a terrorist attack or an accident. All we know is that a plane has crashed into a building. Later on, all we know is that two planes have crashed into buildings. The question after each crash is: how likely is it that a terrorist attack has occurred. –  David Jan 4 '13 at 4:30
1  
Incidentally, an example to show the mathematics involved in Bayes to a non-specialist - assuming a UK population of 60,000,000. Lawyer gets up in court and presents forensic evidence which proves that the match between the accused and the criminal is one in a million - which sounds convincing. However, this means there are 60 people in the population who would match the forensics and our accused is just one of those 60. Unless we have more evidence we can't say the accused is the criminal. –  Mark Bennet Jan 4 '13 at 4:36

5 Answers 5

So we can be almost certain the second plane crash is a terrorist attack, and we can be even more certain that it's accidental?

Correct, there is no contradiction here.

If we know that the first crash was a terrorist attack, then the second crash would be more likely another terrorist attack.

The same reasoning with accidental crashes.

Question 1: Am I correct that Nate Silver is doing it wrong? Question 2: Am I doing it right?

No. There is no need to update the rate of accidental crashes. IMHO, Nate implies that accidental crashes don't include terrorist ones. Otherwise, he couldn't multiply probabilities in the denominator.

share|improve this answer
    
"If we know that the first crash was a terrorist attack" .. but we don't know that. All we know is that there was a first crash. Knowing this, the probability that the first crash was a terrorist attack is not 100%. It's 38%. And the probability that we have just experienced an accidental crash is 61.5%. –  David Jan 4 '13 at 13:59

though the chance of two accidental plane crashes can multiplied to give you a very small number (1/12500 x 1/12500) since they are independent, one cannot assume the same for a terrorist attack. Once we think that the first plane crash is a TPA, it would not make sense to assume that the second crash, if it is also a TPA, to be independent and not highly correlated (perhaps 90% chance that the 2nd plane crash is TPA given the first is TPA) to the first. so if you use 1/20000 x 0.9 to get the probability that both plane crashes are TPA, you will not end up with the problem you mentioned that both scenarios have become more likely.

share|improve this answer

P(TPA_1/PC)=0.38 P(TPA_2/TPA_1)=0.9 (if 1st plane crash is TPA, 2nd plane crash is almost surely TPA since the two events are highly correlated) P(APC_1/PC)=0.62 P(APC_2/APC_1)=1/12500 (on a bright sunny day, accidental plane crashes have to be independent of each other) hence probability that it is a terrorist attack when the 2nd plane crashed= P(TPC_2/TPC_1)P(TPC_1/PC)/ (P(TPC_2/TPC_1) P(TPC1/PC) + P(APC_2/APC_1) P(APC_1/PC) ) = (0.9 X 0.38)/(0.9X0.38+ 0.62X 1/125000)=99%

share|improve this answer

I think there is a misunderstanding of Bayes' theorem here. When you update the probability, you do so because you know for sure what has happened the first time; you are not guessing. In the example, Nate Silver assumes, after the first crash, that it was terrorist caused. With that assumption, you go and update and get a new probability. So, the meaning of the 99.9% is that "IF" the first crash was terrorist caused, then there is almost certainty that if there is a 2nd one, it will also be terrorist caused. Now, if the first was accidental, and later there is a second one, you can also be almost certain that the 2nd was another accident.

share|improve this answer

Has the factor regarding, shall we say, 'level of foreign hostility' been taken into account? Are the numbers calculated according to number of terrorist attacks in country X within a given amount of time?

I live in Canada and if this occurred in Toronto, wouldn't the probability of the plane crashes being terrorist in nature be lower than for America's because we are not generally seen as having antagonized foreign countries to the same degree?

Now imagine if this occurred in Sweden as they're a neutral country and haven't warranted the same amount of potential terrorist repercussions as Canada or USA, then should the probability of the crashes being accidental be factored in?

share|improve this answer
    
This doesn't seem to give an answer to the question; if you have a different question, use the "ask question" link above, and link back to this one. –  user61527 Nov 30 '13 at 0:11

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.