Stationary state of an infinite markov chain

Question

Consider the modified gambler's ruin problem starting with $\$i$ with chance of winning a dollar=$p$ and chance of losing a dollar = $q=1-p$ but, with the condition that when the gambler reaches $\$0$, he either wins a dollar with probability $=p$ or stays at $\$0$ without incurring debt. Then, what is the stationary state of the transition matrix of this infinite markov chain? Also, what can be said about the expectation of the return time to $\$i$. I am quite perplexed by this problem. To know stationary distribution, we need to solve the equation $\pi P=\pi$ which gives me $qx_k=x_k$ or $px_k=x_k$, where$x_k$ is any component of the vector $\pi$. How do we proceed? As for expected return time, do we encounter a infinite geometric series? Can the gambler almost surely win, say $\$100$? Any ideas. Thanks beforehand.

Answer 1

The recursion relation for $x_n^{(t)}$ (the probability of being at position $n$ at time $t$) is $$ x_n^{(t)}=px_{n-1}^{(t-1)}+(1-p)x_{n+1}^{(t-1)} +[n=0](1-p)x_{0}^{(t-1)} $$ with $x_m^{(t)} = 0$ for all $m<0$ and any $t$.

Writing the stationary state as $\{x_n^{(s)}\}$ the recursion relation becomes $$ x_n^{(s)}=px_{n-1}^{(s)}+(1-p)x_{n+1}^{(s)} +[n=0]px_{0}^{(s)} $$ For $n=0$ we can solve this to find $$ x_0^{(s)}=+(1-p)x_{1}^{(s)} +px_{0}^{(s)}\implies x_{1}^{(s)}=x_{0}^{(s)} $$ Now assume that $k>1$ and for all $n<k$, $x_{k}^{(s)}=x_{0}^{(s)}$ (we have just proven the basis case of $k=2$). Consider the recursion relation for $x_{k-1}^{(s)}$: $$ x_{k-1}^{(s)} = px_{k-2}^{(s)}+(1-p)x_{k}^{(s)}\implies x_{0}^{(s)} = px_{0}^{(s)}+(1-p)x_{k}^{(s)}\implies x_{k}^{(s)} = x_{0}^{(s)} $$ So by (strong) induction, for all $k$, $x_{k}^{(s)}=x_0^{(s)}$.

Thus the stationary state has equal probability for every point on the line $n\geq 0$.

This is no more (or less) unsettling than the stationary state of a $p=\frac12$ random walk, which is also equal probabilities for all points on the line.