I need to resolve a gambler's ruin problem so I have been looking into problem number 5 in the following pdf, which is similar to mine and it is resolved:


Consider the gambler’s ruin problem with p = 0.6 and n = 4. Starting in state 2. Find:

First, they create Q, the matrix that describes the probability of transitioning from some transient state to another. What is a submatrix of the transition matrix

$$ Q=\begin{pmatrix} 0 & .6 & 0 \\ .4 & 0 & .6 \\ 0 & .4 & 0 \\ \end{pmatrix} $$

Then, they calculate the matrix that let us know the expected number of times the chain is in state j, given that the chain started in state i.

$$ (I-Q)^{-1}=\begin{pmatrix} 1.4615 & 1.1538 & 0.6923\\ 0.7692 & 1.9231 & 1.1538\\ 0.3077 & 0.7692 & 1.4615 \\ \end{pmatrix} $$


(b) the probability of ever visiting state 1

the probability of ever visiting state 1 is 0.7692/1.4615 = 0.5263

Why is this division supposed to return the probability of ever visiting state 1?

The exercise in the pdf does not have such as section, but I would also need to calculate the expected number of visits to state 3 but in the first 2 transitions. Any hint?

  • 1
    $\begingroup$ For (a), notice that the (2,3) entry of $(I-Q)^{-1}$ is also 1.1538. The question mentions that you start in state 2. $\endgroup$
    – angryavian
    Commented May 2, 2022 at 1:55
  • $\begingroup$ @angryavian My fault. I edited the question. Thanks a lot! $\endgroup$
    – Alfonso_MA
    Commented May 2, 2022 at 2:02

1 Answer 1


Let $V_1$ denote the total number of visits to state 1. Write $E_i(V_1)$ for the mean of $V_1$ when the chain starts at state $i$. Write $A_1$ for the event that the chain ever reaches state 1. Then using the (strong) Markov property at the time of the first visit to state 1, we obtain $$E_2(V_1)=P_2(A_1) E_2(V_1|A_1)=P_2(A_1)E_1(V_1) \,.$$ That explains why $P_2(A_1)$ is the ratio $E_2(V_1)/E_1(V_1)$.


For your second question, using additivity of expectation, the expected number of visits to state 3 in the first two transitions (starting at state 2) is $Q(2,3)+Q^2(2,3)$. In the case of the given chain, the second summand vanishes and you obtain simply $Q(2,3)$.


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .