2
$\begingroup$

I've learned about the most canonical gambler's ruin problems, but what if winning or losing on a previous turn changes the probability of winning or losing on the following turn? Say each turn I either win or lose. Every turn after winning, I win with probability p, and every turn after losing, I win with probability q. How would I compute the long term proportion of turns won. Say I want to approximate the number of turns I won after n turns.

$\endgroup$
2
$\begingroup$

You can study this using a Markov chain with two states (1=post-win, 2 = post-loss), and transition matrix $$ \pmatrix{ p & 1-p\cr q & 1-q\cr}$$ The equilibrium probabilities are $q/(q+1-p)$ and $(1-p)/(q+1-p)$. The long term proportion of wins is then $q/(q+1-p)$.

$\endgroup$
  • $\begingroup$ Besides the asymptotic result, how would you address the question for finite $n$? $\endgroup$ – megas Dec 14 '14 at 21:26
  • $\begingroup$ If $T$ is the transition matrix, $$T^n = \dfrac{1 - (p-q)^n}{1+q-p} T + \dfrac{(p-q)^n - p + q}{1 + q - p} I$$ $\endgroup$ – Robert Israel Dec 14 '14 at 21:48
  • $\begingroup$ Thank you for your response Robert. $\endgroup$ – Janet Dec 15 '14 at 23:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.