A Coin is Tossed Until Heads is Encountered n number of Times Let $C$ be a coin whose probability for showing heads is $p$. Suppose we have an experiment in which we toss the coin till we see heads for the first time.
Let's describe this experiment formally. The sample space $\Omega$ here is the following set $\Omega=\{H, TH, TTH, \ldots \}=\{T^nH: n\geq 0\}$.
We assign a probabilty $P:\Omega\to \mathbf R$ as $P(T^nH)=(1-p)^n p$.
Here's my problem. In order for this to genuinely be a probability measure on $\Omega$, we must have $\sum_{n=0}^\infty P(T^nH)=1$. And an elementary calculation shows that this is indeed the case.
But is there a more conceptual reason as to why is this happening. Defining $P(T^nH)=(1-p)^n p$ is natural because when we toss a coin in succession $n+1$ times, the probability that tails shows up for the first $n$ times and then heads shows up at the end "must be" $(1-p)^np$, for the tosses are "independent". But where, in which probability space, is this independence being talked about?
More generally, if we had a more complicated experiment, where the coin $C$ was tossed till heads shows for a total of $n$ times, then checking that $P$, defined in the similar spirit as for $n=1$, is indeed a probability measure would be quite cumbersome. One can of course think of more involved scenarios.
So basically we want to build a new probability space out of the simple space $\{H, T\}$ with probability measure $P(H)=p$ and $P(T)=1-p$. But is there some conceptual framework which lets us build the spaces above, and more, without the bare hands cumbersome approach?
Thank you.
 A: As I see it, your question is about developing a formal framework in which to study coin tossing problems. To this end, I attempt to answer your question by defining a probability space in which the coin tossing experiment can be studied.

But where, in which probability space, is this independence being talked about?

Let $\Omega = \{ H, T\}^\infty$, the set of all infinite sequences of heads and tails. Make the following definitions:


*

*Let $A = \{ \omega \in \Omega : \omega_a = H \}$ and $B = \{ \omega \in \Omega : \omega_b = H \}$ for some $a ,b \in \mathbb{N}$, where $\omega_i$ is the $i$th element of the sequence $\omega$. Define $A$ and $B$ are independent iff $a \ne b$. This is a way of formalizing that "coin tosses are independence". 

*If $A = \{ \omega \in \Omega : \omega_a = H \}$, then $\mathbb{P}(A) = p$


These two conditions define a probability space that models your scenario well enough to derive from it geometric and binomial random variables.
So in the case of your geometric random variable, define $E_n = \{ \omega \in \Omega : \omega = T^nH\ldots\}$. Define also $E_\infty = \{TTT\ldots\}$, the set containing only the sequence of infinite heads.Then the $E_n$ are mutually disjoint: i.e. $E_i \cap E_j = \emptyset$ for $i \ne j$. Further
$$
\Omega = E_\infty \cup E_0 \cup E_1 \cup E_2 \cup E_3 \cup \cdots
$$
So since the $E_i$ are mutually disjoint, we have
$$
\underbrace{\mathbb{P}(\Omega)}_{=1} = \underbrace{\mathbb{P}(E_\infty)}_{=0} + \sum_{n=0}^\infty \mathbb{P}(E_n) = \sum_{n=1}^\infty \mathbb{P}(E_n)
$$
So $\sum_{n=0}^\infty \mathbb{P}(E_n) = 1$. No algebra required. You can imagine using a similar argument to construct other random variables, such as "where the coin $C$ was tossed till heads shows for a total of $n$ times".
Unfortunately, there's not really a free lunch here. This formalism doesn't really avoid you having to compute complicated sums and derive combinatorial expressions for the probability. What it does do is show that there is, indeed, an ambient probability space in the background which makes it meaningful to speak of "independence of coin tosses" etc.
Not essential, and somewhat technical. Technically, there are actually many different probability spaces that satisfy our two conditions. While these different probability spaces all handle "normal" cases the same, they may treat very weird events differently. Here is one way of defining the probability space for the case of a fair coin.Notice that $\Omega$ can be bijected with $[0,1]$ by treating sequences of heads and tails like binary expansions. E.g. $HTHTHTHTHTHTHT\ldots \mapsto 0.1010101010101010\ldots$. Let $f : \Omega \to [0,1]$ be this bijection. Then let the set of measurable sets $\mathscr{F}$ to be the preimage of Lebesgue-measurable sets in $[0,1]$. Then for $A \in \mathscr{F}$, let $\mathbb{P}(A) = m(f(A))$, where $m$ is the Lebesgue measure. If defined this way, the probability space $(\Omega, \mathscr{F}, \mathbb{P})$ has all the properties you'd want for a sequence of fair coin tosses: coin tosses are independent and the probability of any particular flip being heads is $1/2$. If you're familiar with measure theory, you might find it interesting to think of how to define a probability space for any $p \in (0,1)$.
