# Two related questions about flipping a coin an infinite number of times

So I've been thinking about probability, and frequently you hear people talking about what it means for a coin to come up heads with probability $\frac{1}{2}$. If you were to flip the coin an infinite number of times, then the proportion of heads to tails would be $\frac{1}{2}$. This seems weird to me, and here's why: say we had a special coin that came up heads with probability $\frac{9}{10}$, and we flipped it infinitely many times. This seems to mean that the set of coin flips that come up heads should be $9$ times bigger than the set of coin flips that come up tails. However, both sets are denumerable, so strictly speaking these sets should be the same size—no? Is this not contradictory?

My second question is related: it occurs to me that what is truly meant by "if we let $n$ be the number of coin flips and $p(n)$ be the proportion of $n$ tosses that come up heads, $\lim_{n\rightarrow\infty}p(n)=\frac{9}{10}$" (this still bothers me however because it seems to imply that there's no such thing as an infinite set of coin tosses, and, well, shouldn't there be?), but the $\infty$ that we are approaching here is the countable one. Could we ever imagine flipping a coin once for every real number? It would be like, a function $t:\mathbb{R}\rightarrow \{H,T\}$ with the property that for all $x\in \mathbb R, P(t(x)=H)=\frac{9}{10}$. Same question though: I would want the set $\{x\in \mathbb{R}|t(x)=H\}$ to be somehow nine times bigger than $\{x\in \mathbb{R}|t(x)=T\}$, but they're both infinite (uncountably so, this time). What's going wrong? I've tried asking this in other places, but I got some answers that seemed not to check out to me. You guys seem pretty smart though.

-
For your first question you want the notion of asymptotic density; there’s some discussion and a reference in this answer to a somewhat related question. For your second, the appropriate notion is Lebesgue measure, after you replace $\Bbb R$ by $[0,1]$ (which has the same cardinality). –  Brian M. Scott Apr 15 '12 at 22:27
In essence, the problem is that cardinality is too crude as a measure of "size". –  Nate Eldredge Apr 15 '12 at 22:50
This kind of thing tears apart families, and destroys more lives than you realize. Do not flip a coin an infinite number of times. It's an addiction, pure and simple. Stop after a finite number of coin flips. There are people who love you, who need you. –  Will Jagy Apr 15 '12 at 23:38

You seem to have lots of questions, which I will try to answer, as well as providing comments on certain statements you make.

If you were to flip the coin an infinite number of times, then the proportion of heads to tails would be $\frac{1}{2}$.

That is roughly the right idea, but the problem with this statement is that it isn't precise. What exactly does it mean to flip a coin an infinite number of times, and how can we define the proportion of heads to tails for such a sequence of flips?

This seems weird to me, and here's why: say we had a special coin that came up heads with probability $\frac{9}{10}$, and we flipped it infinitely many times. This seems to mean that the set of coin flips that come up heads should be $9$ times bigger than the set of coin flips that come up tails. However, both sets are denumerable, so strictly speaking these sets should be the same size—no? Is this not contradictory?

This indicates that cardinality of sets isn't a useful way to interpret your first statement above. Fortunately, cardinality isn't the only way to approach the problem of comparing sizes of sets. If we want to make your first statment precise, we will have to come up with another way of defining the "proportion of heads to tails".

My second question is related: it occurs to me that what is truly meant by "if we let $n$ be the number of coin flips and $p(n)$ be the proportion of $n$ tosses that come up heads, $\displaystyle\lim_{n\rightarrow\infty}p(n)=\frac{9}{10}$"

Yes, that is one standard way of interpreting your first statement. More generally, if $S$ and $T$ are subsets of the natural numbers, we can define the proportion of the size of $S$ to the size of $T$ via the limit $$\lim_{n\to\infty} \frac{|S\cap\{1,\ldots,n\}|}{|T\cap\{1,\ldots,n\}|}$$ For example, using this definition, the proportion of even numbers to all numbers is $1/2$. Note, however, that this limit doesn't always exist, so we can't always define the proportion of two sets.

Also, it is not really true in your example that the limit will equal $9/10$. What is true is that the limit is almost surely $9/10$. That is, $$P\left(\lim_{n\to\infty} p(n) = \frac{9}{10}\right) = 1.$$ This is known as the Strong Law of Large Numbers.

This still bothers me however because it seems to imply that there's no such thing as an infinite set of coin tosses, and, well, shouldn't there be?

This objection has to do with the "standard" approach to calculus/analysis using limits. If you think about it, basically the same complaint could be made any time we use limits to define anything.

If we instead use non-standard analysis instead, then the sizes of subsets of $\mathbb{N}$ are well-defined infinite numbers, and the ratio of two such numbers gives the proportion of the sizes of two sets.

However, even though non-standard analysis may be "philosophically" better, it is mathematically worse, because the details of non-standard analysis turn out to be much harder to understand than the details of standard analysis, and they give roughly the same results.

The $\infty$ that we are approaching here is the countable one. Could we ever imagine flipping a coin once for every real number?

As far as I know, mathematicians have never developed a good formalism for answering questions about an uncountable number of coin flips.

My (fairly uninformed) impression is this has to do with the set theory that we have chosen. Intuitive probabilistic arguments involving sets often give contradictory results when coupled with the axiom of choice -- see Freiling's argument, for example. With a different set theory, I (naively) suspect it would be possible to talk about "randomly" chosen subsets of $[0,1]$, and hope to have these sets be measurable. However, I don't know if this is really possible, and even if it were it's not clear that it would have much value beyond the aesthetic and philosophical.

-