# Markov Chain: Using one-step analysis to calculate the probability to reach state $5$ before reaching state $0$ when starting in state$3.$

I have an alternating random walk question where the stochastic process $${X_n : n = 0, 1, 2, . . .}$$ is a Markov chain on the integers with $$X_0 = 0$$. If $$|X_n|$$ is even, the conditional probabilities $$P(X{n+1} = i + 1|X_n = i) = 3/4$$ and $$P(X_{n+1} = i − 1|X_n = i) = 1/4$$ are given, whereas if $$|X_n|$$ is odd, the conditional probabilities are$$P(X_n+1 = i + 1|X_n = i) = 1/4$$ and $$P(X_{n+1}= i − 1|X_n = i) = 3/4.$$ I have been asked to use a one-step analysis to calculate the probability to reach state $$5$$ before reaching state $$0$$ when starting in state $$3$$.

I thought of this question as a Gambler's Ruin problem, but i'm not sure if i'm correct. I defined $$A_j$$ as the event of reaching state $$5$$ starting in state $$j=3$$ before reaching state $$N=0$$. So there are two absorbing states $$5$$ and $$0$$. So $$P(A_3)= \theta_3$$ with $$\theta_5 =1$$ (since probability of reaching 5 when starting at 5 is equal to 1) and using one step analysis i have that $$\theta_3 = 3/4\theta_4 +1/4\theta_2$$ from $$\theta_j = p\theta_{j+1} +q\theta_{j-1}$$ since they're even and from the question i have $$p=3/4$$ and $$q=1/4$$. How should i find $$\theta_2, \theta_4$$? Would greatly appreciate the help

You would find $$\theta_2$$ and $$\theta_4$$ in the same way as you found $$\theta_3$$, so in this case $$\theta_2 = 3/4 \theta_3 + 1/4 \theta_1$$ and $$\theta_4 = 3/4 \theta_5 + 1/4 \theta_1$$. This will give you 4 equations in the 4 unknowns $$\theta_1, \theta_2, \theta_3, \theta_4$$, which you can then solve with linear algebra.