# Discrete-time Markov chain properties

A Markov chain in discrete time is irreducible, has state space $\{0,1,\dots\}$ and starts at $1$. It is both a branching process and a martingale. Determine the probability of hitting $0$.

• what did you try in solving this problem?
– SBF
Commented May 23, 2013 at 13:37
• Where did you get the problem? Why does it interest you? What do you know about Markov chains, about irreducibility, about branching processes, about martingales? Commented May 23, 2013 at 13:42
• Nice question (the answer is $1$). It is a pity that this user is mishandling the site to such an extent...
– Did
Commented Jul 4, 2013 at 17:28

Under the assumption that this is a birth-and-death reversible MC (rate of inflow=rate of outflow for each state), and the transition probabilities are $p,q,r$ (to go up one state/down one state, remain in the same state) with $p+q+r=1$ and boundary value $h_{0,0}=1$ we can get the following equation: $$h_{1,0}=qh_{0,0,}+rh_{1,0}+ph_{2,0}\\ p h_{1,0}=qh_{2,0}$$ The second equation come from the reversibility property (detailed balance equation). Solving these, we get: $$h_{1,0}=\frac{q^2}{p^2-q^2+pq}$$