# Gambler’s Ruin Problem: how to calculate different probabilities?

I need to resolve a gambler's ruin problem so I have been looking into problem number 5 in the following pdf, which is similar to mine and it is resolved:

https://www.webpages.uidaho.edu/~fuchang/StochasticModel/Review3Solution.pdf

Consider the gambler’s ruin problem with p = 0.6 and n = 4. Starting in state 2. Find:

First, they create Q, the matrix that describes the probability of transitioning from some transient state to another. What is a submatrix of the transition matrix

$$Q=\begin{pmatrix} 0 & .6 & 0 \\ .4 & 0 & .6 \\ 0 & .4 & 0 \\ \end{pmatrix}$$

Then, they calculate the matrix that let us know the expected number of times the chain is in state j, given that the chain started in state i.

$$(I-Q)^{-1}=\begin{pmatrix} 1.4615 & 1.1538 & 0.6923\\ 0.7692 & 1.9231 & 1.1538\\ 0.3077 & 0.7692 & 1.4615 \\ \end{pmatrix}$$

...

(b) the probability of ever visiting state 1

the probability of ever visiting state 1 is 0.7692/1.4615 = 0.5263

Why is this division supposed to return the probability of ever visiting state 1?

The exercise in the pdf does not have such as section, but I would also need to calculate the expected number of visits to state 3 but in the first 2 transitions. Any hint?

• For (a), notice that the (2,3) entry of $(I-Q)^{-1}$ is also 1.1538. The question mentions that you start in state 2. Commented May 2, 2022 at 1:55
• @angryavian My fault. I edited the question. Thanks a lot! Commented May 2, 2022 at 2:02

Let $$V_1$$ denote the total number of visits to state 1. Write $$E_i(V_1)$$ for the mean of $$V_1$$ when the chain starts at state $$i$$. Write $$A_1$$ for the event that the chain ever reaches state 1. Then using the (strong) Markov property at the time of the first visit to state 1, we obtain $$E_2(V_1)=P_2(A_1) E_2(V_1|A_1)=P_2(A_1)E_1(V_1) \,.$$ That explains why $$P_2(A_1)$$ is the ratio $$E_2(V_1)/E_1(V_1)$$.
$$----$$
For your second question, using additivity of expectation, the expected number of visits to state 3 in the first two transitions (starting at state 2) is $$Q(2,3)+Q^2(2,3)$$. In the case of the given chain, the second summand vanishes and you obtain simply $$Q(2,3)$$.