Countable vs. Uncountable Probability In Apostol's Calculus II, he splits the calculus of probability into two chapters:


*

*Finite and Countable sets

*Uncountable sets


He only remarks that explaining why different tools are needed for uncountable sets "would take us too far afield."
I have an intuition for why a probability measure defined over a finite set would have different requirements than one defined over an infinite set, but I'm not really sure why uncountable sets are so different.
I suspect it may have something to do with the first theorem of the chapter (if $S$ is an uncountable set, there are an most a countable number of $s\in S$ where $P(s)\not=0$), but I'm not sure what exactly. 
A search online doesn't seem to turn up much, so maybe this is just a decision the author made to separate them that others don't agree with? More likely I just don't know the terms to search for.
 A: Let us suppose first that the sample space $\Omega$ is finite or countably infinite. Then $\Omega=\{\omega_1, \omega_2, \cdots, \omega_n\}$ for some positive integer $n$, or $\Omega=\{\omega_1,\omega_2,\omega_3,\dots\}$. In either case, if we know the $p_i=P(\omega_i)$, then for any subset $S$ of $\Omega$, we can find $P(S)$ by summation (finite summation, or summation of an infinite series). 
Suppose now that $\Omega$ is uncountable. Then, as you mentioned, there are at most countably many $\omega$ such that $P(\omega)\ne 0$. We can separate out these, but will be typically left with an uncountable subset $\Omega' \subseteq \Omega$ such that $P(\Omega')\ne 0$. Indeed in the majority of standard situations, $P(\omega)=0$ for all $\omega$. For uncountable subsets $S\subset \Omega$, we will therefore not be computing $P(S)$ by a summation. In the usual situations (normal distribution, exponential distribution, many others both univariate and multivariate), computations of probabilities and related quantities such as expectations are done by integration.
So there is a large difference between the finite or countably infinite case on the one hand, and the uncountable case on the other, in the tools that we use. Typically it is summation versus integration. One could say that summation is a special form of integration. But that does not make the distinctions collapse entirely. Typically, for technical reasons, not all subsets of $\Omega$ can be assigned probabilities in the uncountable case. 
A: One essential difference is that it is possible in some cases to sum a finite or countably infinite number of positive real numbers and have a result of $1$.  So these numbers can be probabilities of disjoint events.
It is never possible to do the same with the sum of an uncountable number of positive real numbers. 
