# Problems with infinite $\Omega$, when trying to define product spaces of discrete probability spaces

Definitions In our course we defined a discrete probability space as a tuple $\left(\Omega,P\right)$, where $P:\mathcal{P}(\Omega)\rightarrow\left[0,1\right]$ and $\Omega$ is at most countable, such that $P\left(\Omega\right)=1$ and for a finite our countable infinite set $I$ we have $P\left(\bigcup_{i\in I}A_{i}\right)=\sum_{i\in I}P\left(A_{i}\right)$ where all $A_{i}$ are pairwise disjoint.

Motivation Now we did several exercise that implicitly involved product spaces. For all these exercises $\Omega$ was finite.

Problem To avoid having to construct for each example separately a product space, I wanted to do it once and for all in the abstract, by proving the following theorem: Let $\left(\Omega_{1},P_{1}\right),\ldots,\left(\Omega_{n},P_{n}\right)$ be discrete probability spaces. Then $\left(\Omega_{1}\times\ldots\times\Omega_{n},P\right)$ where $P:\Omega_{1}\times\ldots\times\Omega_{n}\rightarrow\left[0,1\right],\ P\left(\omega_{1}\ldots\omega_{n}\right)=P_{1}\left(\omega_{1}\right)\cdot\ldots\cdot P_{n}\left(\omega_{n}\right)$ is a discrete probability space as well.

But in trying to prove the second property ($\sigma$-addivity) of discrete probability spaces, I ran into problems: How do I prove, that $$P\left(\bigcup_{i\in I}A_{i}\right)=P_{1}\left(\text{pr}_{1}\left[\bigcup_{i\in I}A_{i}\right]\right)\cdot\ldots\cdot P_{n}\left(\text{pr}_{n}\left[\bigcup_{i\in I}A_{i}\right]\right)=$$ $$=\sum_{i\in I}P_{1}\left(\text{pr}_{1}\left[A_{i}\right]\right)\cdot\ldots\cdot\sum_{i\in I}P_{n}\left(\text{pr}_{n}\left[A_{i}\right]\right)\quad\left(\star\right)$$

equals $\sum_{i\in I}\left(P_{1}\left(\text{pr}_{1}\left[A_{i}\right]\right)\cdot\ldots\cdot P_{n}\left(\text{pr}_{n}\left[A_{i}\right]\right)\right)=\sum_{i\in I}P\left(A_{i}\right)$ ? ($\text{pr}_{j}\left[\cdot\right]$ denotes the $j$-th projection mapping)

If the $\Omega_i$'s are finite I can prove my theorem, since together with the disjointness of the $A_{j}$'s, this implies that $I$ has to be finite, so everything is fine. But for countable infinite $\Omega_i$'s, the sum from $\left(\star\right)$ can only be evaluated using the Cauchy product for series, and this number wouldn't be the same as $\sum_{i\in I}\left(P_{1}\left(\text{pr}_{1}\left[A_{i}\right]\right)\cdot\ldots\cdot P_{n}\left(\text{pr}_{n}\left[A_{i}\right]\right)\right)$, I think.

Can product spaces maybe be defined in a meaningful way only under the conditions from above ?

-
Yes, product measures always exist, even for non-discrete spaces. This should be in almost every measure theory textbook, probably in the chapter that discusses Fubini's theorem. – Nate Eldredge Apr 19 '12 at 21:10
Sigma-additivity is not what you write as $(\ast)$. – Did Apr 19 '12 at 22:34
@NateEldredge Yes, but I'm just doing a course in elementary probability theory and know nothing of measure theory, so I don't even know what a measure is - let alone a product measure. It is no use to me, to know that some objects exist, that are probably an abstractization of probability space $(\Omega,P)$, that give something like want I want. All I want is to know, whether there is a meaningful way for countable infinite $\Omega_i$'s, to get a probability space (see my definition), that is defined in the cartesian product of the omegas. – MyCatsHat Apr 20 '12 at 10:50
@Didier I thought it were "$P\left(\bigcup_{i\in I}A_{i}\right)=\sum_{i\in I}P\left(A_{i}\right)$ where all $A_{i}$ are pairwise disjoint sets". If not, what is it then ? – MyCatsHat Apr 20 '12 at 10:51
@user26698 What you wrote in the comment is correct; but this is not what equation $\star$ says. Suppose $\Omega=\{0,1\}\times\{0,1\}$ and let $A=\{(0,0),(1,1)\}$. You don't get the measure of $A$ using projections. – Byron Schmuland Apr 20 '12 at 12:18

What you did right: You have correctly defined the product space $\Omega=\Omega_1\times\cdots\times \Omega_n$, and assigned probability values to individual points $\omega=(\omega_1,\dots,\omega_n)$ by the product $$P(\omega)=P_1(\omega_1)\cdots P_n(\omega_n).$$ This assigns a non-negative value to every point $\omega\in\Omega$ such that $\sum_{\omega\in\Omega}P(\omega)=1$.
Where you went wrong: Your equation $(\star)$ seems to indicate that you want to assign a probability to an arbitrary subset $A$ of $\Omega$ using projections, as in (2). This is incorrect; you should simply define $$P(A)=\sum_{\omega\in A}P(\omega).\tag1$$ The formula $$P\left( A\right)=P_{1}\left(\text{pr}_{1} A\right)\cdots P_{n}\left(\text{pr}_{n} A\right)\tag 2$$ works only for product sets, but this is a completely separate issue from countable additivity.
What to do next: You want to show that $P$ is countably additive. At this point, you can completely forget about the product structure of $\Omega$. In general, if $\Omega$ is any countable space, and $P$ assigns non-negative values to the points of $\Omega$, where $\sum_{\omega\in \Omega}P(\omega)=1$, then (1) defines a countably additive measure on all subsets of $\Omega$. The proof of this will involve manipulating countably many infinite sums, but I think you can do it.
Ok, I think I understood where my problem was: First I defined a discrete probability model where $P$ was actually defined on $\mathcal{P}(\Omega)$ and not just $\Omega$ (this was a typo of mine, which I now corrected), and then, when establishing the product space, I switched to defining the space in a different manner, namely via $P$ being defined directly on $\Omega_{1}\times\ldots\times\Omega_{n}$. Wanting to prove the $\sigma$-additivity I mixed the two definitons and wanted to show $\sigma$-additivity (...) – MyCatsHat Apr 22 '12 at 8:20
(...) like in the first probability definition - which, as you said, isn't even necessary, since every mapping $P$ that satisfies $(1)$ gives me a mapping $P'$ that is $\sigma$-additive. But if I had defined my $P$, as it is used in $(\star)$, like this: $$P:\mathcal{P}(\Omega_{1})\times\ldots\times\mathcal{P}(\Omega_{n})\rightarrow \left[ 0,1 \right]$$ $$P\left(A_{1}\ldots A_{n}\right)=P_{1}\left(\text{pr}_{1}\left[A_{1}\right]\right)\cdot\ldots\cdot P_{n}\left(\text{pr}_{n}\left[A_{n}\right]\right)$$, then I think $(\star)$ would have correctly been the $\sigma$-additivity. – MyCatsHat Apr 22 '12 at 8:34
What do you think ? Is that correct ? And how would I go under this definition (i.e. without using your indicated route to define $P$ like $P\left(\omega\right)=P_{1}\left(\omega_{1}\right)\cdot\ldots\cdot P_{n}\left(\omega_{n}\right)$ and then using a theorem that tells me that whenever a function $P:\Omega \rightarrow \left[0,1\right]$ satisfies $(1)$ I get a function $P'$ on $\mathcal{P}(\Omega_{1})\times\ldots\times\mathcal{P}(\Omega_{n})$ - which obviously works) to prove in the more direct way the sigma additivity (i.e. proving $(\star)$) ? – MyCatsHat Apr 22 '12 at 8:44