A Markov chain in discrete time is irreducible, has state space $\{0,1,\dots\}$ and starts at $1$. It is both a branching process and a martingale. Determine the probability of hitting $0$.

  • 2
    $\begingroup$ what did you try in solving this problem? $\endgroup$
    – Ilya
    May 23, 2013 at 13:37
  • 1
    $\begingroup$ Where did you get the problem? Why does it interest you? What do you know about Markov chains, about irreducibility, about branching processes, about martingales? $\endgroup$ May 23, 2013 at 13:42
  • $\begingroup$ Nice question (the answer is $1$). It is a pity that this user is mishandling the site to such an extent... $\endgroup$
    – Did
    Jul 4, 2013 at 17:28

1 Answer 1


Under the assumption that this is a birth-and-death reversible MC (rate of inflow=rate of outflow for each state), and the transition probabilities are $p,q,r$ (to go up one state/down one state, remain in the same state) with $p+q+r=1$ and boundary value $h_{0,0}=1$ we can get the following equation: $$ h_{1,0}=qh_{0,0,}+rh_{1,0}+ph_{2,0}\\ p h_{1,0}=qh_{2,0} $$ The second equation come from the reversibility property (detailed balance equation). Solving these, we get: $$ h_{1,0}=\frac{q^2}{p^2-q^2+pq} $$

  • $\begingroup$ Your question was to determine the probability of 'hitting 0' without providing any specific details. This is this probability under some specific assumptions on the MC $\endgroup$
    – Alex
    May 23, 2013 at 16:25
  • $\begingroup$ Offtopic. The hypotheses made in this answer contradict the assumption that the process is both a branching process and a martingale. $\endgroup$
    – Did
    Jul 4, 2013 at 17:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.