5
$\begingroup$

I am taking some courses about stochastic processes, Markov chains, ... and there we need basic measure-theoretic probability theory.

My background is computer science, where I had an introductory course about probability and statistics from a quite applied point of view. Moreover, almost all I had up to now was discrete probability theory. Additionally, we had almost nothing in analysis, so my knowledge there is also quite modest.

Now I am confronted with expressions like "almost surely", "$\sigma$-algebra", "measurable space", "measurable function", and so on.

I've read some definitions about $\sigma$-algebras and so on, so together with my background of discrete probability theory I know a little bit what they are talking about but I am missing the essentials.

So I plan to learn all the basics for probability theory and measure theory at some point, but now my goal is to learn all the important things I need to know to follow the courses in Stochastic Processes in a more or less short time.

Do you know a "friendly" book covering these topics at a somehow "superficial" level such that a non-mathematician can follow it and at the same time provides all the essential things to understand stochastic processes (from an applied point of view)?

Thank you very much.

EDIT: I know that there are other questions asking for measure theory books. But I am searching for a somehow easier book for non-mathematicians.

$\endgroup$
7
$\begingroup$

I like the book Probability, Statistics, and Random Processes for Electrical Engineering by Leon-Garcia (I used it to teach probability last semester). Of course, it is Electrical Engineering focused. It supresses the esoteric measure theory material.

The title of your book is funny because my first reaction is "Who but a mathematician would ever need measure theory anyway?" Of course I am being a bit snarky with that comment. I do think it is important for advanced students to at least know what measure theory is about. However, I feel it is more important for students to know the difference between a countably infinite set and an uncountably infinite set. Knowing the difference is essential for measure theory, but unfortunately some courses skip this, assume you already know it, and cover the less essential topics of sigma algebras.

In my humble opinion, you don't really need measure theory for probability or stochastic processes, although measure theory is certainly the foundation for those things. Similarly, Russell's Principia Mathematica is a foundation for basic arithmetic, but most people who use arithmetic (including mathematicians) have never read it (and certainly arithmetic existed before it). So, it is possible for you to learn and use something without going into detail on foundations. It is also good to know those foundations exist, especially if you eventually want answers to lingering questions.

The term "almost surely" is the most important one that you listed. It is synonymous with "with probability 1." For example, if you have a random variable $X$ that is uniformly distributed over the interval $[0,1]$, then $Pr[X \neq 1/2]=1$ and so almost surely $X$ is not $1/2$. Similarly, since the rational numbers in $[0,1]$ can be listed as $\{q_1, q_2, q_3, \ldots\}$ we have:

$$ Pr[\mbox{$X$ is rational}] = \sum_{i=1}^{\infty}Pr[X=q_i]=0 $$

and so $Pr[\mbox{$X$ is irrational}]=1$, which means $X$ is almost surely irrational.


Some sets are so complicated that they cannot have probabilities assigned to them. The term measurable describes a set that has a valid probability. A theorem that says "assuming the set is measurable" is just being precise, and you can ignore that phrase without worry. It just means they are restricting to the case where probabilities are defined (you cannot prove theorems otherwise). All the crazy theorems about measurability are designed to basically show that all practical sets of interest are measurable. In that sense, measure theory is self-defeating, since its most important results ensure you can safely ignore them.


Perhaps the most practical topics in measure theory are the convergence theorems, such as the Lebesgue dominated convergence theorem. These theorems tell you when you are allowed to pass a limit through an integral or through an expectation.

A "sigma algebra" is a class of sets defined so that all sets in the class can be measured. The "standard Borel sigma algebra" is a very large class of sets of real numbers. It is always possible to define the probability that a random variable falls in one of those sets. Since those sets are so extensive, every practical set you will ever work with will indeed be measurable (unless you end up working on foundational mathematical subjects or axiom-of-choice related set theory subjects).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.