I am reading somewhat simultaneously on Bayesian probability theory (from Jaynes' probability theory and the logic of science), and on the measure-theoretic approach to probability.

Now, I casually overheard a Bayesian (it may have been Jaynes) say that measure theory as a basis for probability is "ad hoc".

So this left me wondering: what is the logical relation between the measure-theoretic approach and Bayesian probability theory?

  • Are they two completely distinct ways of building up probability theory?
  • Can measure theory be used as a tool, given the Bayesian framework?
  • something else?

What you're referring to as Bayesian probability theory is more commonly referred to as probability theory as derived from Cox's theorem:


The measure-theoretic approach is built using the Kolmogorov axioms (https://en.wikipedia.org/wiki/Probability_axioms), whereas Cox's theorem uses certain desiderata about consistent reasoning to derive the rules of probability theory. They merely represent two different derivations of the exact same practical tools; the sum and product rules, for example, are equally applicable regardless of which approach you're using.

However, there are some subtle reasons why Jaynes was so picky about deriving probability theory via Cox's theorem, rather than the Kolmogorov axioms.

If you're reading Jaynes's Probability Theory: The Logic of Science, you'll notice that his opening chapters discuss two big flaws (in Jaynes's opinion) with the Kolmogorovian approach. First, Jaynes believed that probability theory should be able to quantify the certainty of propositions, rather than merely observable events. It's possible to use the Kolmogorov approach for this purpose, but it's not explicitly built from the ground up for it and occasionally gets messy (your "measurable events" essentially become "observable and distinguishable universes"). Cox's theorem, on the other hand, is constructed solely to reason about propositions--hence Jaynes's strong preference.

Second, Jaynes felt that infinite sets (which are often used in the Kolmogorov approach) were completely incoherent unless the limiting behavior that generated those sets from an original finite set were specified. Since Cox's theorem never uses any fancy set theory to derive the basic rules of probability, this problem is avoided altogether. If I remember correctly, however, this benefit has some unfortunate drawbacks, and is kind of a double-edged sword--I vaguely recall reading that probability as derived from Cox's theorem only possesses finite additivity, but not countable additivity, whereas the measure-theoretic derivation possesses both.

So to conclude, the Kolmogorov axioms and Cox's theorem represent completely distinct ways of building up probability theory. Yes, measure theory can be used as a tool given the Bayesian framework, but with some aforementioned caveats.

As for the Bayesian you overheard describing the measure-theoretic approach as "ad-hoc," I'd say that's a little harsh. In mathematics, it's always a blessing when we're able to establish the same results from multiple different approaches. The fact that probability theory is derivable from two totally different starting points gives us a lot more flexibility when it comes to formal proofs and practical applications alike. We can have a preference for one framework over another, but they're both still really cool.

This is a really deep topic, and if you want to learn more I'd recommend talking to your local mathematics professor. Best of luck!


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.