0
$\begingroup$

I’m developing a tool to calculate the probability of conclusions using facts and Bayes Rule. I’ve encountered quite a puzzle. The problem is that when I use two different forms of Bayes Rule and estimated variables I get two different results. Let me try to explain.

Form A - The standard form of Bayes Rule is:

P(conclusion | fact) = P(conclusion) x P(fact | conclusion) / P(fact).

Form B - P(fact) must be based on prior data. If P(fact) is not available, then the no prior data form of Bayes Rule can be used. This is:

P(conclusion | fact) = P(conclusion) x P(fact | conclusion) / (P(conclusion) x P(fact | conclusion) + (1 – P(conclusion)) x P(fact | not conclusion))

Let’s use an example. The fact (the effect) is that a graph shows a clear national GDP growth long-term trend after a recession was quickly corrected. The conclusion (the cause) is the long-term GDP growth trend is due to good management of economic policy. This gives us five variables of interest. All but the first are estimated.

  1. P(conclusion | fact) = unknown. This is what we want to calculate.

  2. P(conclusion) = .01. We start with a low prior probability so that we can accumulate evidence that P(conclusion | fact) is highly likely to be true.

  3. P(fact) = .95. All data points to a clear strong trend. No data is contested.

  4. P(fact | conclusion) = .99. The probability rises from .95 to .99 given the conclusion.

  5. P(fact | not conclusion) = .05. If the conclusion is false, then the fact is thrown into high doubt. It somehow cannot be true at all. .05 represents extremely unlikely.

Plugging these five variables into the two forms of Bayes Rule, we get

  • Form A: .01 x .99 / .95 = .01

  • Form B: .01 x .99 / (.01 x .99 + ((1 - .01) x .05)) = .167

The two results, .01 and .167, do not agree. They are not even close. Why is this?

The discrepancy can be eliminated two ways, by changing estimates to calculated values using the other variables.

The first way to eliminate the discrepancy is to change P(fact | not conclusion) from .05 to .95. But that makes no sense. .95 says the fact is still highly true. But it cannot be highly true if the conclusion is false. A false conclusion would say “the long-term GDP growth trend is NOT due to good management of economic policy.” Therefore, it must be due to something else. What would that be? BAD economic policy? Luck? Sudden stimulus due to war? None of the alternative reasons make any sense, because in this case (the reaction by the US administration to the 2008 Great Recession) they did not occur, though that could be controversial. What I’m trying to explain is that estimates for P(fact | not conclusion) for this example will tend to be low.

The second way to eliminate the discrepancy is to change P(fact) from .95 to .06. But that also makes no sense. It says the fact is almost completely false. But the data trend is clear and uncontested. There’s no way it can be false.

Of interest is that when the two forms of Bayes Rule are used on problems where the variables can be set by counting in a sample space, there is no discrepancy. This makes me suspect that Bayes Rule works well for sampling logic, but does poorly for some types of general logic, where some variables must be estimated.

Form A’s result make little sense. The prior and the posterior probabilities are the same. That’s like ignoring an extremely useful fact. Reducing the estimate for P(fact) to increase the posterior doesn’t make sense, because that implies the less true the fact, the more true the conclusion.

Form B’s result makes the most sense. It agrees with intuition. Is this because when using general logic to estimate variables, general logic works best with form B and not form A?

Thanks.

$\endgroup$
17
  • $\begingroup$ You think a certain fact is almost surely true, yet unless a highly unlikely conclusion also happens to be true, the fact is very unlikely. That is a contradiction. At least one of your probability estimates is based on faulty reasoning. $\endgroup$
    – David K
    Commented Sep 28, 2018 at 21:13
  • $\begingroup$ Thanks. David. Can you explain your comment? The way I see it, the fact that a graph of GDP data shows a recession was quickly corrected and the trend continued for a long time has a HIGH probability of truth. Also, why is the conclusion highly unlikely? It is routine for administrations to take corrective action when recessions occur. $\endgroup$
    – JHarich
    Commented Sep 28, 2018 at 22:25
  • $\begingroup$ I did not say the conclusion was unlikely. You said that. Possibly you are completely misusing Bayes; I really cannot follow your logic. $\endgroup$
    – David K
    Commented Sep 29, 2018 at 0:04
  • $\begingroup$ Thanks. What part of my logic is hard to follow? I can try to explain it better. $\endgroup$
    – JHarich
    Commented Sep 29, 2018 at 0:09
  • $\begingroup$ "We start with a low prior probability so that we can accumulate evidence that P(conclusion | fact) is highly likely to be true." Why? The usual application of Bayes is that we choose the prior based on our best prior estimate. We don't deliberately choose a low prior just to show improvement. $\endgroup$
    – David K
    Commented Sep 29, 2018 at 0:22

1 Answer 1

2
$\begingroup$

P(FACT) =(P(conclusion) x P(fact | conclusion) + (1 – P(conclusion)) x P(fact | not conclusion)) by law of totaly probability https://en.m.wikipedia.org/wiki/Law_of_total_probability so it is not 0.95

$\endgroup$
3
  • $\begingroup$ Thanks papasmurfete. Yes, I know that. It gives P(fact) = .06. That's how form B of Bayes Rule calculated the answer without having to know P(fact). My question is why, given plausible estimates that make intuitive sense, do we get such different results from the two forms of Bayes Rule. Form A uses variables 2, 3, and 4. Form B uses variables 2, 4, and 5. P(fact) is variable 3. $\endgroup$
    – JHarich
    Commented Sep 28, 2018 at 22:45
  • $\begingroup$ Both forms of Bayes' rule have P(fact) in the denominator. The only difference is that in the "no prior data" form, P(fact) has been expressed in other terms using the law of total probability. $\endgroup$
    – David K
    Commented Sep 29, 2018 at 0:09
  • $\begingroup$ Furthermore think that: From Form A variables 2,3,4: youve got that P(fact | not conclusion))= [P(fact)-P(fact|conclusion)xP(conclusion) ]/P(not conclusion)=[0.95-0.01x0.99]/0.99=0.949. But from Form B: you get P(fact | not conclusion))= 0.05, so one of your way of gaining your info for your variables is fault or probably false interpreted. $\endgroup$ Commented Sep 29, 2018 at 7:37

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .