28
$\begingroup$

I'd like to have a correct general understanding of the importance of measure theory in probability theory. For now, it seems like mathematicians work with the notion of probability measure and prove theorems, because it automacially makes the theorem true, no matter if we work with discrete and continuous probability distribution.

So for example, expected value - we can prove the Law of large numbers using its general definition (measure theoretic) $\operatorname{E} [X] = \int_\Omega X \, \mathrm{d}P$, and then derive the formula for discrete and continuous cases (discrete and continuous random variables), without having to prove it separately for each case (we have one proof instead of two). One could say that the Law of large numbers justifies the definition of expected value, by the way.

Is it right to say that probability using the general notion of probability measure saves work of mathematicians? What are the other advantages?

Please correct me if I'm wrong, but I hope you get the idea of what sort of information I expect - it's the importance and role of measure theory in probability and answer to the question: are there theorems in probability that do not hold for general probability measure, but are true only for either discrete or continuous probability distribution? If we can prove that no such theorems can exist, we can simply forget about the distinction between discrete and continuous distributions.

If someone could come up with a clear, concise summary, I'd be grateful. I'm not an expert, so please take that into account.

$\endgroup$
9
  • 2
    $\begingroup$ The probability mass function of a discrete random variable is the density with respect to counting measure over the sample space. So in the measure-theoretic sense, there isn't really a distinction between discrete and continuous random variables (aside from the choice of measure). $\endgroup$
    – Math1000
    Nov 1, 2015 at 3:11
  • $\begingroup$ To be more explicit, the density of a random variable $X$ that takes values in a countable subset $N\subset\mathbb R$ is the Radon-Nikodym derivative of $X^{-1}\circ \mathbb P$ with respect to counting measure on $N$. $\endgroup$
    – Math1000
    Nov 1, 2015 at 4:13
  • 2
    $\begingroup$ Not that I can think of at the moment, no. Why the interest? $\endgroup$
    – Did
    Nov 1, 2015 at 9:37
  • 1
    $\begingroup$ Consider the theorem: "Let $(\Omega, \mathcal{A}, \mathcal{P})$ be a probability space, where $\Omega$ is countable. Then there is an $\omega\in\Omega$ with $\mathcal{P}[\{\omega\}] > 0$". This "theorem" is obviously false for non-countable $\Omega$'s. So, here you have it, a measure-theoretic theorem that holds only for discrete spaces. Essentially it says: discrete spaces are discrete, which is quite boring. However, strictly speaking, all theorems are merely tautologies anyway, so you would have to define what you mean by "important theorems". $\endgroup$
    – Andrey Tyukin
    Nov 10, 2015 at 16:55
  • 2
    $\begingroup$ For the sake of completeness, also read this answer about E.T. Jaynes and his "outright dismissal of any measure theory". $\endgroup$
    – Han de Bruijn
    Nov 11, 2015 at 12:45

3 Answers 3

Reset to default
14
+50
$\begingroup$

Since measure-theoretic axiomatization of probability was formulated by Kolmogorov, I think you'd be very much interested in this article. I had similar questions to you, and most of them were clarified after the reading - although I've also read Kolmogorov's original work after that.

One of the ideas is that historically there were proofs for LLN and CLT available without explicit use of measure theory, however both Borel and Kolmogorov started using measure-theoretical tools to solve probabilistic problems on $[0,1]$ and similar spaces, such as treating a binary expansion of $x\in [0,1]$ as coordinates of a random walk. Then the idea was: it works well, what if we try to use this method much more often, and even say that this is the way to go actually? When the work of Kolmogorov was first out, not every mathematician was agree with his claim (to say the least). But you are somewhat right in saying that measure theory allows dealing with probability easier. It's like solving basic geometric problems using vector algebra.

Regarding facts exclusively available for discrete/continuous distributions: usually a good probabilistic theorem is quite general and works fine with both cases. However, there are some things that hold for "continuous" measures only. A proper name for continuous is atomless: $\mu$ is atomless if for any measurable set $F$ there exists $E \subseteq F$ such that $\mu(E) < \mu(F)$ where inequality must be strict. Then the range of $\mu$ is convex, that is for all $0 \leq c \leq \mu(\Omega)$ there exists a set $C$ such that $\mu(C) = c$. Of course, that does not hold for measures with atoms. Not a very probabilistic fact though.

$\endgroup$
3
  • $\begingroup$ I think the most important part of my question is whether there are theorems in probability that are true only for either discrete or continuous probability distribution. $\endgroup$
    – user4205580
    Nov 17, 2015 at 9:52
  • 1
    $\begingroup$ @user4205580: the idea of what sort of information I expect - it's the importance and role of measure theory in probability. If you think that another part of your question is more important, please explicitly mention this in the question itself. $\endgroup$
    – SBF
    Nov 17, 2015 at 10:41
  • $\begingroup$ @user4205580: edited $\endgroup$
    – SBF
    Nov 17, 2015 at 12:01
14
$\begingroup$

2 reasons why measure theory is needed in probability:

  1. We need to work with random variables that are neither discrete nor continuous like $X$ below:

Let $(\Omega, \mathscr{F}, \mathbb{P})$ be a probability space and let $Z, B$ be random variables in $(\Omega, \mathscr{F}, \mathbb{P})$ s.t.

$Z$ ~ $N(\mu,\sigma^2)$, $B$ ~ Bin$(n,p)$.

Consider random variable $X = Z1_A + B1_{A^c}$ where $A \in \mathscr{F}$, discrete or continuous depending on A.

  1. We need to work with certain sets:

Consider $U$ ~ Unif$([0,1])$ s.t. $f_U(u) = 1_{[0,1]}$ on $([0,1], 2^{[0,1]}, \lambda)$.

In probability w/o measure theory:

If $(i_1, i_2) \subseteq [0,1]$, then $$P(U \in (i_1, i_2)) = \int_{i_1}^{i_2} 1 du = i_2 - i_1$$

In probability w/ measure theory:

$$P(U \in (i_1, i_2)) = \lambda((i_1, i_2)) = i_2 - i_1$$

So who needs measure theory right? Well, what about if we try to compute

$$P(U \in \mathbb{Q} \cap [0,1])?$$

We need measure theory to say $$P(U \in \mathbb{Q} \cap [0,1]) = \lambda(\mathbb{Q}) = 0$$

I think Riemann integration doesn't give an answer for $$\int_{\mathbb{Q} \cap [0,1]} 1 du$$.

Furthermore, $\exists A \subset {[0,1]}$ s.t. $P(U \in A)$ is undefined.

From Rosenthal's A First Look at Rigorous Probability Theory:

enter image description here

enter image description here

enter image description here

enter image description here

$\endgroup$
4
  • $\begingroup$ Dear anonymous downvoter, OP said 'it's the importance and role of measure theory in probability.' How have I not introduced precisely that? I based my answer on class notes and Rosenthal's book $\endgroup$
    – BCLC
    Nov 15, 2015 at 22:08
  • $\begingroup$ Oh wait I think I made a mistake in the last line. A is I guess supposed to be Borel but actually it's ok if it isn't. Perhaps it's supposed to be P(A) $\endgroup$
    – BCLC
    Nov 23, 2017 at 5:43
  • 1
    $\begingroup$ Did you mean $\mathbb{Q}\color{red}{\cap}[0,1]$? $\endgroup$
    – Snoop
    Dec 8, 2022 at 12:31
  • $\begingroup$ @Snoop lol yes thanks! $\endgroup$
    – BCLC
    Dec 8, 2022 at 21:17
5
$\begingroup$

There's an exciting and amazing theorem in Kai Lai Chung's A Course in Probability Theory about distribution functions (d.f.), which states :

enter image description here

or with this refinement,

enter image description here This super professional statement, overkills every old fashion probability theory about discrete, continuous or mixed distribution functions !

Theorem 1.3.2 can't be proven, unless by the powerful paradigm, Measure Theory.

Moreover, Measure Theory has much more tools to study Probability Theory. In fact,

  • Strong Law of Large Numbers can NOT be proven without measure Theory.

  • You can NOT define Brownian Motion precisely

  • You can NOT work with Stochastic Differential Equations without Measure Theory
$\endgroup$
3
  • 1
    $\begingroup$ If I've not understood what you want, please correct mo to enhance the answer. $\endgroup$
    – Fardad Pouran
    Nov 15, 2015 at 17:53
  • 1
    $\begingroup$ Don't see what's the problem w/ your answer or mine $\endgroup$
    – BCLC
    Nov 15, 2015 at 22:04
  • 2
    $\begingroup$ +1; this is a very cool result! $\endgroup$
    – Yatharth Agarwal
    Mar 13, 2019 at 2:26

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .