Question
Let $(X_n)_{n\ge0}$ be a Markov Chain with stochastic matrix $P$, determine whether or not the state $0$ is recurrent when $p_1=p_2<0.5$ and $\gamma >0$. The stochastic matrix P is given by:
![]()
My Thoughts
Intuitively, the state $0$ seems to be recurrent.
I know that since we have one single irreducible communicating class, this means that if I can show that any of the states are recurrent, then this automatically implies that the state $0$ is also recurrent.
If I can show that for some $i \in \{{0,1,2...}\}$ that $P(T_i< \infty :X_0=i)=1$ then this is sufficient to show that a state $i$ is recurrent (where $T_i$ denotes the first passage time of the state $i$ starting from the state $i$.
I know that we are able to form recurrence relations to compute quantities such as expected hitting time or hitting probabilities, and I am wondering if I am able to use a similar argument in this case.
I would be grateful for any guidance. On a finite state space, these concepts are much clearer to me, as we can use the fact that a communicating class is closed to derive recurrence. However, this argument does not hold for infinite state spaces.