I'm going gambling and I have ten dollars. I have a gambling device that costs 1 dollar per game to play. I win with a probability of $\frac{1}{5}$ and each win gives me back four dollars (to a net profit of three) while lost games give nothing back for a net loss of one dollar.

I'm going to play the game until I go bankrupt (net wins and net losses sum up to $-10$ or less) or I make a profit of five dollars (net wins and net losses sum up to $5$ or more). I want to know the probability of the game ending in me winning.

At first I tried analyzing this using the binomial distribution but I'd need to know the total number of games in advance, but it is unbounded. Ideas on how to approach this are warmly welcome.

  • 5
    $\begingroup$ Do you know anything about Markov chains? They are the perfect tool to deal with problems like this. For example, you are asking for the "absorption probabilty" of $+5$. $\endgroup$ – Marc Feb 16 '16 at 9:16
  • $\begingroup$ @Marc, yeah, but I've only used them to create cheesy chatbots. Never studied them in the more mathematical sense. Thanks! $\endgroup$ – kviiri Feb 16 '16 at 9:17
  • 1
    $\begingroup$ Take a look at this document: statslab.cam.ac.uk/~james/Markov/s13.pdf. Theorem 1.3.2 is basically what you need and example 1.3.1 shows how to use it. $\endgroup$ – Marc Feb 16 '16 at 9:19
  • $\begingroup$ Other than Markov chains and recurrent equations, this is a problem suitable for theory of martingales. I did a few calculation and I get 42.54% as a probability the game ends up with you winning. See if it matches other methods results. $\endgroup$ – Kolmo Feb 16 '16 at 13:08
  • 1
    $\begingroup$ @kviiri i found a mistake in my method and a simulation I performed returned your estimate. I'll keep thinking $\endgroup$ – Kolmo Feb 16 '16 at 18:14

As stated in the comments I will use the theory of Markov Chains. We work on the Markov chain $\{-10,-9,\ldots,6,7\}$, where the numbers represent the possible profits/losses. Probabilities are as described in the problem. Let $X_n$ define the state of the Markov chain at time $n$ and, define $A = \{5,6,7\}$ and \begin{align} H^A(\omega) &= \inf\{n\ge0\mid X_n(\omega)\in A\}\\ h_i &= \mathbb{P}(H^A<\infty\mid X_0 = i) \end{align} The number we are interested in is $h_0$, the chance of ever hitting the set $A$, given that we start at $0$. We use theorem 1.3.2 of this link which tells us that $h_i$ are the smallest solution to the linear system $$ \begin{bmatrix} 1&0&0&0&0&0&0&0&0&0&0&0&0&0&0&0&0&0\\ 4&-5&0&0&1&0&0&0&0&0&0&0&0&0&0&0&0&0\\ 0&4&-5&0&0&1&0&0&0&0&0&0&0&0&0&0&0&0\\ 0&0&4&-5&0&0&1&0&0&0&0&0&0&0&0&0&0&0\\ 0&0&0&4&-5&0&0&1&0&0&0&0&0&0&0&0&0&0\\ 0&0&0&0&4&-5&0&0&1&0&0&0&0&0&0&0&0&0\\ 0&0&0&0&0&4&-5&0&0&1&0&0&0&0&0&0&0&0\\ 0&0&0&0&0&0&4&-5&0&0&1&0&0&0&0&0&0&0\\ 0&0&0&0&0&0&0&4&-5&0&0&1&0&0&0&0&0&0\\ 0&0&0&0&0&0&0&0&4&-5&0&0&1&0&0&0&0&0\\ 0&0&0&0&0&0&0&0&0&4&-5&0&0&1&0&0&0&0\\ 0&0&0&0&0&0&0&0&0&0&4&-5&0&0&1&0&0&0\\ 0&0&0&0&0&0&0&0&0&0&0&4&-5&0&0&1&0&0\\ 0&0&0&0&0&0&0&0&0&0&0&0&4&-5&0&0&1&0\\ 0&0&0&0&0&0&0&0&0&0&0&0&0&4&-5&0&0&1\\ 0&0&0&0&0&0&0&0&0&0&0&0&0&0&0&1&0&0\\ 0&0&0&0&0&0&0&0&0&0&0&0&0&0&0&0&1&0\\ 0&0&0&0&0&0&0&0&0&0&0&0&0&0&0&0&0&1 \end{bmatrix} \mathbf{x} = \begin{bmatrix} 0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\0\\1\\1\\1 \end{bmatrix} $$ I know it looks big and dangerous, but with a program like eg matlab we can easily solve this to get to the solution. I don't have a Gaussian elimination program available at the moment but the numerical solution gives \begin{align} h_{-10} & = 0.0000 \\ h_{-9} & = 0.0188 \\ h_{-8} & = 0.0403 \\ h_{-7} & = 0.0652 \\ h_{-6} & = 0.0939 \\ h_{-5} & = 0.1266 \\ h_{-4} & = 0.1648 \\ h_{-3} & = 0.2087 \\ h_{-2} & = 0.2571 \\ h_{-1} & = 0.3179 \\ h_{0} & = 0.3840 \\ h_{1} & = 0.4510 \\ h_{2} & = 0.5608\\ h_{3} & = 0.6486\\ h_{4} & = 0.7189\\ h_{5} & = 1.0000\\ h_{6} & = 1.0000\\ h_{7} & = 1.0000 \end{align}

Note that an advantage of this method is that you get the hitting probabilities for all possible starting values.

  • $\begingroup$ Decided to accept this answer because of the theory link you provided in the comments plus actually showing me the linear equation system. Thanks a lot! $\endgroup$ – kviiri Feb 18 '16 at 10:55

Just a setup.

So definitely not a full answer, but too much for a comment.

For $n\in\mathbb{Z}$ let $p_{n}$ denote the probability that the game ends in winning if your credit is $n$.

Then $p_{n}=0$ if $n\leq0$ and $p_{n}=1$ if $n\geq15$.

For $n\in\left\{ 1,\dots,14\right\} $ we have the relation:


You are looking for $p_{10}$.

  • $\begingroup$ This looks promising, I'll run through the calculations (I might need a lot of paper for this one!) $\endgroup$ – kviiri Feb 16 '16 at 10:04
  • $\begingroup$ You can also use a program to solve the system of linear equations. $\endgroup$ – Marc Feb 16 '16 at 16:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.