# Discrete-time Markov chain properties

A Markov chain in discrete time is irreducible, has state space $\{0,1,\dots\}$ and starts at $1$. It is both a branching process and a martingale. Determine the probability of hitting $0$.

• what did you try in solving this problem?
– Ilya
May 23, 2013 at 13:37
• Where did you get the problem? Why does it interest you? What do you know about Markov chains, about irreducibility, about branching processes, about martingales? May 23, 2013 at 13:42
• Nice question (the answer is $1$). It is a pity that this user is mishandling the site to such an extent...
– Did
Jul 4, 2013 at 17:28

Under the assumption that this is a birth-and-death reversible MC (rate of inflow=rate of outflow for each state), and the transition probabilities are $p,q,r$ (to go up one state/down one state, remain in the same state) with $p+q+r=1$ and boundary value $h_{0,0}=1$ we can get the following equation: $$h_{1,0}=qh_{0,0,}+rh_{1,0}+ph_{2,0}\\ p h_{1,0}=qh_{2,0}$$ The second equation come from the reversibility property (detailed balance equation). Solving these, we get: $$h_{1,0}=\frac{q^2}{p^2-q^2+pq}$$