1
$\begingroup$

The Third Axiom of Probability has been a well-accepted axiom yet there seems to be a nice proof of it. I couldn't trace an implicit assumption of this axiom anywhere in the proof. I'll write a sketch of the proof; please tell me if it could be replaced by it.

Thm : Let $\{E_1, E_2, \cdots\}$ be mutually exclusive events of some countable set $\Omega$ such that $P(\Omega) = 1$. Then $P(E) = \displaystyle\sum_{i=1}^{\infty} P(E_i)$.

Proof : The case when $E$ is finite can be proven easily (from definition) and hence is left out. (Edit : Added as comment #1) When $E$ is countable, choose some $\epsilon > 0$. Denote $\displaystyle\sum_{i=1}^{\infty} P(E_i)$ by $\text{P}$ and $F_n = \displaystyle\sum_{i=1}^{n} P(E_i)$.

Observe that we can get some $n$ in $\mathbb{N}$ such that $|P - F_n| \leq \epsilon$ (which basically follows from the monotone convergence thm and def. of convergence). Additionally, we can yield some (finite set) $E^{*}_i \subseteq E_i$ for each $1 \leq i \leq n$ so that $P(E_i) \leq P(E^{*}_i) + \epsilon$.

Obviously $\displaystyle\bigcup_{i=1}^{n} E_i^{*} \subseteq E$ and so by Boole's inequality implies $P\left(\displaystyle\bigcup_{i=1}^{n} E_i^{*}\right) \leq P(E)$. From all these above, it follows that : $$\text{P} \leq F_n + \epsilon \leq P\left(\displaystyle\bigcup_{i=1}^{n} E_i\right) + \epsilon \leq \displaystyle\bigcup_{i=1}^{n} P(E_i) + \epsilon \leq \bigcup_{i=1}^{n} P(E_i^*) + (n+1)\epsilon \leq P(E) + \epsilon'$$.

This shows that $\text{P} = \displaystyle\sum_{i=1}^{\infty} P(E_i) \leq P(E)$. We can also show the reverse to eventually conclude the equality of $\text{P}$ and $P(E)$. (I'm not attaching the proof, unless asked in comments to keep the size of the post decent)


Why is it an axiom and not a theorem then?

I would like to add that I have no background in measure theory and hence is not defined on a measure theoretic space in particular.

EDIT (as suggested by @Bungo in comments) : Please note that the $\Omega$ is assumed to be countable. I'm trying to see if setting $P(A \cup B) = P(A) + P(B)$ as the axiom ($A \cap B = \phi; A,B \subseteq \Omega$) proves both the finite and the infinite case.

$\endgroup$
  • 11
    $\begingroup$ Your "proof" of the finite case implicitly assumes that $P(E) = \sum_{\omega \in E}P(\omega)$. But how would this follow from the first two axioms ($P(E) \geq 0$ and $P(\Omega) = 1$)? Moreover, it's not even true in general, e.g. if $E$ is uncountably infinite. $\endgroup$ – Bungo Feb 14 at 19:11
  • 5
    $\begingroup$ 1. You baselessly assume convergence. 2. Your final line says $P\le P(E)+(n+1)\varepsilon$, it seems that the $n$ depends on the $\varepsilon$; so you can't conclude much from that. $\endgroup$ – Thorgott Feb 14 at 19:15
  • 8
    $\begingroup$ To see that any attempt to "prove" this axiom is fruitless, consider the following probability "distribution", which satisfies axioms 1 and 2 but not 3, therefore you can't prove 3 from 1 and 2. Let $\Omega = \{a, b\}$, and let $P(\emptyset) = P(\{a\}) = P(\{b\}) = P(\Omega) = 1$. $\endgroup$ – Bungo Feb 14 at 19:20
  • 6
    $\begingroup$ @charlesh Why do you assume that for disjoint $A,B$ we have $P(A \cup B) = \sum_{\omega \in A \cup B} P(\omega)$? This does not follow in any way from the first two axioms. Moreover, even assuming all three axioms, it is not true if $A \cup B$ is uncountable or if any of the singletons $\{\omega\} \subseteq A \cup B$ are not measurable. $\endgroup$ – Bungo Feb 14 at 19:32
  • 5
    $\begingroup$ This won’t answer your question nor explain why your theorem will or will not work, but do note that that countable additivity of the probability measure can be proven if we assume what some authors call left continuity of measures as the third axiom instead: if $A_n \supset A_{n+1}$ is a decreasing sequence of events with $\cap A_n =\emptyset$ then $P(A_n)\to 0$ as $n\to \infty$. In fact one can prove $P$ is left continuous if and only if $P$ is countably additive. For context, Kolmogorov took left continuity as an axiom and proved countable additivity when he axiomatized prob. theory. $\endgroup$ – Nap D. Lover Feb 14 at 20:04
4
$\begingroup$

Why is [countable additivity] an axiom and not a theorem then?

Countable additivity of a probability measure can be proven as a theorem if we assume what some authors call left continuity of measures as the third axiom instead: if $A_n \supset A_{n+1}$ is a decreasing sequence of events with $\cap_n A_n=\emptyset$ then $\lim_{n\to \infty} P(A_n)\to 0$.

In fact one can prove $P$ is left continuous if and only if $P$ is countably additive. For context, Kolmogorov took left continuity as an axiom and proved countable additivity when he axiomatized probability theory in 1933 (and if I recall correctly, showed the equivalence as well). Ignoring matters of pedagogy, efficiency of proofs, and applications, it is then merely a matter of taste which to pick as the axiom and which to derive as the theorem, since they are equivalent. It seems most modern texts choose countable additivity as the axiom, however.

I'm trying to see if [finite additivity axiom can be used] to prove the infinite case

All we must do to show this is impossible is find one probability measure on a countable sample space $\Omega$ that is finitely additive but not countably additive, and then we know that finite additivity cannot, in general, imply countable additivity. This is apparently not trivial to do even on $\Omega = \mathbb{N}$, as it requires, in some form, the axiom of choice or ultra-filters. See here, here and here for more details about the set-theoretic issues and the examples using ultra-filters, the natural density, and/or AC. In summary, the counterexamples we want exist but cannot be explicitly constructed.

Please comment for clarification or if I've made any mistakes!

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Right. Makes much more sense now. So basically this mostly happens when the limit does not exist (I'm only considering the space to be countable). If the limit exists and we have the base case (P(A U B) = P(A) + P(B) for disjoint A,B) as the axiom, the finite and infinite cases can be derived from there. Is that true? $\endgroup$ – charlesh Feb 15 at 14:43
  • $\begingroup$ The base case can be extended by induction, yes, but countable additivity follows directly from left continuity. You do not need to assume anything further, like the finite additive base case. I can add proof of the equivalence if you’d like (it is not measure theory technical). Finally, if there is some decreasing sequence of events ($A_n\supset A_{n-1}$ with $\cap_n A_n=\emptyset$) that does not have a vanishing limit, then left continuity fails, and by equivalence, so does countable additivity. $\endgroup$ – Nap D. Lover Feb 15 at 18:57
  • $\begingroup$ Okay, the proof I mentioned above for countable additivity seems to work. Could you please tell me the flaw in it? I still cannot figure out why that won't prove the countable additivity case upon assumption of the base case. $\endgroup$ – charlesh Feb 16 at 11:01
  • $\begingroup$ @charlesh I agree with user Thorgott’s last comment. Since your chosen $n=n(\epsilon)$ depends on $\epsilon$, there is no guarantee $(n+1)\epsilon$ is small. As $\epsilon \to 0$, we may have to pick substantially larger $n$ and its not necessary for $(n+1)\epsilon$ to stay small. Also probably irrelevant but in the last big inequality line I think there are some typos: it does not make sense to bound $P(\cup E_i)+\epsilon$ by $\cup P(E_i)+\epsilon$ you probably meant to write $P(\cup E_i)+\epsilon \leq \sum P(E_i) +\epsilon$ (subscripts omitted for brevity). $\endgroup$ – Nap D. Lover Feb 16 at 19:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.