Sigma Algebra on Coin Toss

I am trying to study probability space, and so far I have come to point that probability space is defined as $( \Omega, \mathcal{F}, P )$ where $\mathcal{F}$ is the $\sigma-algebra$. I know what $\sigma-algebra$ is, but I am confused that the $\sigma-algebra$ can be easily obtained by the power set of $\Omega$ .i.e. $2^\Omega$. Since a set can have many $\sigma-algebras$, are we only interested in the smallest $\sigma-algebra$ generated by events of interest?

For example, consider tossing a coin twice, and we are interested in finding the probability such that there is at least one head. So, surely $\Omega = (HH, HT, TH, TT)$. But what about $\mathcal{F}?$ One straight forward option is power set, but I am interested in "smallest" part of $\sigma-algebra$ which seems to come very handy in probability theory? Suppose I take $\mathcal{F} = (\emptyset, \Omega, \{HH, HT\}, \{TH, TT\})$. Now with this $\mathcal{F}$, I cannot measure "at least one head" because $(HH,HT, TH)$ is not in $\mathcal{F}$. It definitely means that my $\mathcal{F}$ has to be chosen smartly. How can I help myself here?

• Please follow the edited version. I updated my $\sigma-algebra$ now. – Wasiq Noor Jul 13 '18 at 10:33
• Note that for countable $\Omega$ it is perfectly reasonable to take power set as sigma algebra, but once it’s uncountable, we can’t because it gets too large. This is what Borel showed if you are interested in further details. Hence, we consider the smallest sigma algebra, usually generated by a collection of sets of interest. – James Yang Jul 13 '18 at 11:19

A $\sigma$-algebra is a collection of sets that needs to satisfy certain conditions. Yours violates one of the most important ones, namely, the fact that it should be closed under countable unions. (If you have a finite $\Omega$ and you put all the single elements into your algebra, then it necessarily has to be the power set by this axiom.)

• Please comment regarding the update in question. – Wasiq Noor Jul 13 '18 at 10:34
• why is it mostly written in books that $\sigma-algebra$ has to be the smallest. What does it mean to have the smallest one? – Wasiq Noor Jul 13 '18 at 10:37
• It means you add the minimum number of sets to satisfy the $\sigma$-algebra condition, a $\sigma$ algebra need not be minimal of course. – E-A Jul 13 '18 at 10:40

A sigma algebra must be closed for countable unions and relative complements.

One such set with that property is the power set of $\Omega$, ie: the set of all subsets for $\Omega$, and when $\Omega=\{{\sf TT, HT, TH, HH}\}$ then that is: \begin{align}\mathcal F & = 2^\Omega\\ & =\{\{\}, \{{\sf TT}\},\{{\sf HT}\},\{{\sf TH}\},\{{\sf HH}\},\{{\sf TT, HT}\},\ldots\{{\sf TH, HH}\},\ldots,\{{\sf TT, HT, TH, HH}\}\}\end{align}

I am not going to list all $16$ members of the powerset.   However, one of them will be $\{{\sf HT, TH, HH}\}$ .

$~\\~\\~$

There is a smaller sigma-algebra over $\Omega$ which contains $\{{\sf HT, TH, HH}\}$, and that is: $$\{\{\},\{{\sf TT}\},\{{\sf HT, TH, HH}\},\{{\sf TT, HT, TH, HH}\}\}$$

If you choose to take a $\sigma$-algebra that is not the power set $\sigma$-algebra, you will necessarily lose information. To see this, let $\Omega = \{x_1, ..., x_n\}$ be a finite set. If $\mathcal{F}$ is a $\sigma$-algebra on $\Omega$ where $\{x_i\} \in \mathcal{F}$ for all $\{x_i\}$, the requirement that $\sigma$-algebras be closed under countable union forces $\mathcal{F} = \mathscr{P}(\Omega)$. Hence, if you choose any $\sigma$-algebra that is not $\mathscr{P}(\Omega)$, you can no longer determine the likelihood of individual outcomes.

However, if you already know the sets in $\Omega$ that you want to be able to measure, then there is a well-defined, smallest $\sigma$-algebra that contains these sets. If $\{E_i\}$ is a collection of subsets of $\Omega$, then the power set $\sigma$-algebra on $\Omega$ is a $\sigma$-algebra containing all $\{E_i\}$. Since the intersection of any number of $\sigma$-algebras on $\Omega$ containing all the $\{E_i\}$ is a $\sigma$-algebra containing all the $\{E_i\}$ the intersection of all $\sigma$-algebras containing the $\{E_i\}$ is a $\sigma$-algebra containing all the $\{E_i\}$ and is clearly the smallest such $\sigma$-algebra. We call this the $\sigma$-algebra generated by the $\{E_i\}$. Qualitatively, this is the $\sigma$-algebra formed by taking complements, finite intersections, and countable unions of members of $\{E_i\}$ until it satisfies all requirements of a $\sigma$-algebra.

In your example, you want to be able to measure the probability of having at least one head. The only set you want to measure in this is now $\{HT, TH, HH\}$. When we go through this process, we obtain the $\sigma$-algebra $$\mathcal{F} = \{\emptyset, \{TT\}, \{HT, TH, HH\}, \{HH, HT, TH, TT\}$$ as the smallest $\sigma$-algebra that makes $\{HT, TH, HH\}$ measurable.

• So to summarize, what you are saying is: We might have situation where we can take into our considerations a particular $\sigma-algebra$ but since likelihood on elementary events is also possible we must take power set as out $\sigma-algebra$. Please correct me if I am wrong. – Wasiq Noor Jul 13 '18 at 11:58
• Yes. If you want to be able to consider all possible elementary outcomes in a finite (or countable) state space, you must take the power set $\sigma$-algebra. If you know in advance the sets you want to be measurable though, there is a well-defined way of finding the smallest such satisfactory $\sigma$-algebra, and this $\sigma$-algebra is often smaller. – Alex Nolte Jul 13 '18 at 12:05