# Flip a coin until a head comes up. Why is “infinitely many tails” an event we need to consider?

Suppose we're considering the problem of flipping a coin until "heads" comes up. The sequences H, TH, TTH, TTTH, ... are all part of the sample space we need to consider.

But what about the sequence TTT... (infinitely many tails)? I would think that this "event" should be excluded by definition, because our experiment doesn't end until "heads" comes up. Nonetheless, one could imagine a scenario where "heads" never comes up.

So when should we include TTT... in our sample space, and in doing so, what does that "buy" us?

-

Added: In this answer I take the sample space to be the full "Bernoulli space" $\mathcal{B} = \{T,H\}^{\infty}$. As Qiaochu Yuan points out in a comment to my other answer, this is a valid way to go: we identify a finite sequence $T^n H$ with the subset of infinite sequences with initial segment $T^n H$, and with respect to the natural (product) measure on $\mathcal{B}$, this event has probability $2^{-n-1}$, as it should. This is a more complicated space than the discrete space appearing in my other answer, but it is arguably more natural: the other space was contrived to answer a very specific problem about Bernoulli trials, whereas this space is the space of all Bernoulli trials.

You could ask the same question about any element of your sample space: "Why should we bother to have $(H,T,T,H,T,H,H,H,T,\ldots)$ in it -- after all, it only occurs with probability zero."

In this case, every singleton set occurs with probability zero, but if you took them all out you would have no sample space!

Based on your other recent question, I think you should start learning about measures and countable additivity. This has been the mathematical underpinning of probability theory for almost $80$ years.

Note also that the Bernoulli space $\mathcal{B}$ has a beautiful structure -- it can be viewed as $(\mathbb{Z}/2\mathbb{Z})^{\infty}$ and thus endowed with both a group structure and a compatible topology, under which it becomes a compact, totally disconnected abelian topological group. You wouldn't just start pulling points out of a compact topological group, would you? That will ruin everything...

-
But the game ends after the first Head, so TTT... is the only event with zero probability. –  TonyK May 25 '11 at 8:23

In my opinion it buys us nothing. Let the random variable $X$ be the number of tosses until the first head. If the probability of a tail is anything other than $1$, the sum of the probabilities of $X=n$, as $n$ ranges over the positive integers, is equal to $1$. So "infinitely many tails," if we were interested in it, would have probability $0$. Since it has probability $0$, and all other possible values of $X$ have non-zero probability, we can just forget about this "possibility."

-
Just because an event has probability zero is no cause to throw it out. Consider the experiment of choosing a random number from [0,1]. EVERY outcome has probability zero (it is just one point belonging to a continuous interval). In this experiment, throwing out probability zero events means throwing out EVERY event. –  Austin Mohr May 25 '11 at 1:18
How in the world does every event have non-zero probability? -1 –  Brennan Vincent May 25 '11 at 1:54
@Austin Mohr: Yes, there are continuous distributions. This one happens to be discrete. –  André Nicolas May 25 '11 at 4:22
@Brennan I meant to say every elementary event has probability zero (the event containing just a singleton). –  Austin Mohr May 27 '11 at 22:28

The number of coin tosses until the first head is not a well-defined notion, because of that singular event. Compare it to "the number of coin tosses until the last head".

On the other hand, you could restrict yourself to the set of samples where this random variable $X$ is finite, and then you could ask $\mathbb{E}[X|B]$, where $B$ is the event "$X<\infty$". Since $\Pr[B]=1$, we can ignore the conditioning without effecting the result.

More formally, the r.v. $X$ cannot really attain the value $\infty$. Instead, you need to define it somehow on the tails sample. The exact way won't effect anything since this is a zero-probability event. In fact, usually r.v. are only defined "up to measure zero" anyhow. So the value at the tails sample is "canceled out".

-

I had previously written an answer which got a fair number of votes. In retrospect, this was a nice answer, but not to this question! (Added: I changed my mind again after reading Qiaochu Yuan's comment. Apologies for leaving two somewhat lengthy answers to the same question. Maybe it will be "educational"?)

The point here is that the sample space here is much simpler than the "Bernoulli space" of countably infinite $\{H,T\}$-valued sequences. This sample space consists of elements $H,TH,TTH,\ldots,T^n H,\ldots$ -- i.e., one element for each $n \in \mathbb{N}$, together with -- perhaps -- the one infinite sequence of all tails.

In particular this sample space is countably infinite. If one is interested only in countably infinite sample spaces then there is no need to bring out measure-theoretic ideas: it is enough to enumerate the elements of the space and assign each element a non-negative real numbered probability in such a way so that the sum of all of these numbers is $1$. Here the element $T^n H$ -- i.e., first $n$ tails, then heads -- should be assigned the probability $\frac{1}{2^{n+1}}$. If we sum over all these singleton probabilities we get $\sum_{n=0}^{\infty} \frac{1}{2^{n+1}} = 1$.

In particular, if we want to put the last point $TTTTT\ldots$ (infinitely many tails) into our space, then we need to assign it probability zero. If you have a discrete probability space then it's true that you don't lose anything by omitting singleton elements of probability zero, any more than you lose anything by omitting zero terms from an infinite series. On the other hand you don't have to omit it: it's legal to put it in and assign it probability zero. Although it won't make any mathematical difference, I would favor doing this from a modelling perspective, because after all flipping a coin infinitely many times and having it come up tails every time is not logically impossible: it just happens that the probability we want to assign to it is zero.

-
One can still interpret this question as a question about the space of all possible coin flips by suitably modifying the events: the event $T^n H$ is just the union of all sequences which begin $T^n H$ in the larger space (the point being that for all finite $n$ there are uncountably many such sequences). –  Qiaochu Yuan May 25 '11 at 11:07
@Qiaochu: how are you distinguishing between two sequences that begin $T^n$H? (otherwise, the collection {TH, TTH,...} is countably-infinite.) –  gary May 25 '11 at 12:36
@gary: I don't understand the question. I'm not. –  Qiaochu Yuan May 25 '11 at 12:39
@Qiaochu: yes, that's true. Maybe I'll undelete my first answer?! :) –  Pete L. Clark May 25 '11 at 17:54