St. Petersburg Paradox

Question

A fair coin will be tossed until a heads results. You will then be paid $2^{n-1}$ dollars where $n$ equals the number of flips. Now why is the expected pay out infinite? $$ \sum_{n \geq 1} (\frac{1}{2^n})2^{n-1} = \sum_{n \geq 1} \frac{2^{n-1}}{2^n} $$ Why does this give the payout? The payout shouldn't be infinite, it should be a potential infinity. For example, if I flip the coin once and it is a tails, then I will receive two dollars for certain. On the second flip there is a $25$ percent chance that I will receive a minimum of $4$ dollars. but a $75$ percent chance that I will receive only two dollars. And so on as the chances of receiving more money decreases and eventually approaches $0$.
Why then is it suggested that an investor pay any amount of money for such an opportunity?

Source: A Brief History of the Paradox, Philosophy and Labyrinths of the Mind

Obviously this appears false by intuition, but I'd like to know the argument for the paradox; why one would pay a trillion dollars to enter such a game. The argument just seems far too fallacious to me.

How many times would one have to play? After almost twenty thousand plays, we are still making less than 8 dollars. IF every twenty thousand plays we increased our winnings by 8 dollars, we would have to play hundreds of billions of times just to start to have an average earnings equal to one trillion... we'd run out of time in the universe it seems...

Edit I am curious, however, if the overall slope of the simulation data would remain consistent, if so why? If not, why? What accounts for the vertical jumps we see occurring less and less often? Why do they occur at all?

Answer 1

Suppose you pay a trillion dollars to enter the game. The following table contains some of the values of the net payoff you can possibly end up with and the corresponding probabilities:

\begin{align*} \begin{array}{rr} 1/2&-\,1\mathord{,}000\mathord{,}000\mathord{,}000\mathord{,}000\\ 1/4&-\,999\mathord{,}999\mathord{,}999\mathord{,}998\\ 1/8&-\,999\mathord{,}999\mathord{,}999\mathord{,}996\\ 1/16&-\,999\mathord{,}999\mathord{,}999\mathord{,}992\\ \vdots\\ 1/2^{10}&-\,999\mathord{,}999\mathord{,}999\mathord{,}488\\ \vdots\\ 1/2^{20}&-\,999\mathord{,}999\mathord{,}475\mathord{,}712\\ \vdots\\ 1/2^{30}&-\,999\mathord{,}463\mathord{,}129\mathord{,}088\\ \vdots\\ 1/2^{40}&-\,450\mathord{,}244\mathord{,}186\mathord{,}112\\ 1/2^{41}&99\mathord{,}511\mathord{,}627\mathord{,}776\\ 1/2^{42}&1\mathord{,}199\mathord{,}023\mathord{,}255\mathord{,}552\\ \vdots\\ 1/2^{50}&561\mathord{,}949\mathord{,}953\mathord{,}421\mathord{,}312\\ \vdots\\ 1/2^{100}&633\mathord{,}825\mathord{,}300\mathord{,}114\mathord{,}114\mathord{,}699\mathord{,}748\mathord{,}351\mathord{,}602\mathord{,}688\\ \vdots\\ 1/2^{200}&803\mathord{,}469\mathord{,}022\mathord{,}129\mathord{,}495\mathord{,}137\mathord{,}770\mathord{,}981\mathord{,}046\mathord{,}170\mathord{,}581\mathord{,}301\mathord{,}261\mathord{,}101\mathord{,}496\mathord{,}890\mathord{,}396\mathord{,}417\mathord{,}650\mathord{,}688\\ \vdots \end{array} \end{align*}

The paradox lies in the following observation. If you take out a loan of one trillion dollars to play this game, you will go bankrupt with a very large probability. However, once in a lifetime (not even of a human but of the universe) you win an unspeakably large amount of money, so large you can't even imagine.

In the light of this observation, a reasonable person would never play this game only once. It is worth playing only if you can play it indefinitely, while you have access to unlimited borrowing. What will happen is that you will keep playing for billions of years, accumulating an enormous debt using your infinite line of credit. But after a very long time, you will win so much money that is sufficient for you to pay off this large debt and still purchase the whole world. As @IanColey put it, this is because the chances of winning so much money are very, very tiny, but the payoffs associated with these very, very tiny probabilities are much, much, much more enormous than the probabilities are tiny.

Answer 2

Expectations are statements about behavior of a random variable as you draw from it infinitely many times. If you had an infinite bankroll, you could play the game infinitely many times at $1,000,000,000,000 a pop, winning money in the long run (even if you lost 10 times in a row). At the same time, we are also assuming (rather incorrectly) that the bankroll of the person handing out the reward is infinite. If either of these assumptions is invalidated, new dynamics enter the equation. If you don't have infinite money, you must consider risk (e.g. if the bet is as big as your entire net worth, you have a 50/50 of losing it all in one toss), and if the person offering the game has finite money, the expectation of playing the game to infinity cannot be infinity since the game ends (as well as your opportunity to accrue new winnings) when he runs out of money.

Wikipedia has a small overview of the mathematical reason it is infinity as well as the problems I just brought up - as well as alternative explanations.

http://en.wikipedia.org/wiki/St._Petersburg_paradox

Answer 3

The game describes a random variable $X$, which represents the amount of money you'll get by playing the game. You don't know what the outcome will be, that's why $X$ is a random variable. We do know however what the values of $X$ may be, namely, anything of the form $2^n$. We also know what is the probability you'll get $2^{n-1}$, namely it is precisely $\frac{1}{2^{n}}$. Thus $P(X=2^{n-1})=\frac{1}{2^{n}}$.

Now, this is of course an ideal game since in reality you will never ever get $2^{100000000000000000000} $ dollars. not because the probability of that happening is very small, but simply because no such amount of money exists. Nonetheless, we may contemplate the properties of this ideal game and deduce properties of any approximation of it, namely when actually playing this game.

The expectation of $X$ (which is $\infty $ by the computation you quote) is a mathematical entity that represents, in some sense, what we expect the value of $X$ to be. This is very very crude though and needs to be interpreted correctly, especially when the expectation is $\infty $.

In the case above, the expectation being $\infty $ means the expected gain from playing the game is $\infty$, roughly in the following sense. Suppose that in order to play the game you have to pay a finite amount $K$ for each time you want to play. If you play long enough (may be a very very long time!) than your total wins will be greater that your total losses, no matter what $K$ is (as long as it is fixed for the entire duration of the game). In this sense, it is mathematically justified to be willing to pay any fixed amount of money to play the game.

Of course the idealistic nature of the mathematical situation neglects various humanly important factors such as utility, which is commonly how this paradox is solved.

Remark: The two envelope paradox, which is a bit related, is much more difficult to resolve.

Answer 4

If the bankroll of your opponent is finite, rather than infinite, the result is quite different. Suppose instead your opponent says that he will pay you $2^{n-1}$ dollars if you flip $n$ tails before your first head, as long as it's less than a trillion dollars (because that's all he has on hand). Then your expected payoff is only $$ 1\cdot\frac{1}{2} + 2\cdot\frac{1}{4}+\ldots+2^{K-1}\cdot\frac{1}{2^K}=\frac{1}{2}K, $$ where $$ K=\lceil\log_{2}10^{12}\rceil=40, $$ and you should only be willing to pay $\$20$ for a chance to play. Similarly, unless you'll really derive twice as much joy from two trillion dollars as from one trillion dollars (which I think would make you unusual), you shouldn't put in more than about $20$ bucks.

The point is that all of the infinite expected return comes from the high end of the payoff distribution, where your intuition tends to break down.