I am working with a specific birth and death chain, defined as follows.

Consider a set of states $X = \{0,1,2,...,n\}$, where $x^* \in (0,n)$ is a recurrent state. Transition probabilities are defined as follows:

$P_{i,j} = \left\{\begin{array}{lc} 1-p_i & \text{ if } i = j+1 \\ p_i & \text{ if } i = j \\ 0 & \text{ otherwise}\end{array}\right., \forall i,j \neq \{0,x^*,n\} \in X.$

Given that $x^*$ is a recurrent state, $P_{x^*,x^*}=1$. Moreover, $P_{0,0}=0$, and also $P_{n,n}=0$.

I am calculating the expected hitting time starting from any state $x < x^*$ (i.e., the hitting time from below). I call this value $\eta_{x,x^*}$ in line with standard notation on Markov Chains.

Starting from the constraint that $\eta_{x^*,x^*} =0$, I retrieved $\eta_{x^*-1,x^*} = \frac{1}{1-P_{x^*-1,x^*-1}}$, and then

$\eta_{x^*2,x^*} = \frac{1}{1-P_{x^*-2,x^*-2}} + \frac{P_{x^*-2,x^*-1}}{(1-P_{x^*-1,x^*-1})(1-P_{x^*-2,x^*-2})},$

$\eta_{x^*-3,x^*} = \frac{1}{1-P_{x^*-3,x^*-3}} \left(1 + \frac{P_{x^*-3,x^*-2}}{1-P_{x^*-2,x^*-2}} + \frac{P_{x^*-3,x^*-2}P_{x^*-2,x^*-1}}{(1-P_{x^*-1,x^*-1})(1-P_{x^*-2,x^*-2})}\right).$

Proceeding backward, I came up with this general formula for any state $x < x^*$:

$$ \eta_{x,x^*} = \frac{1}{1-P_{x,x}} \left(1 + \sum_{i=x}^{x^*-2}\prod_{l=x}^i\frac{P_{l,l+1}}{1- P_{l+1,l+1}} \right), $$

Can someone tell me if this formula is correct?

Thank you in advance, I am happy to give more details in case they are needed.

  • $\begingroup$ Note that I just deleted the second part of the question which is more problematic, and so, I could ask it in a second question $\endgroup$ Commented Nov 20, 2023 at 16:11

1 Answer 1


Your Markov chain has the property that from $i$ you either go to $i+1$ or just stay there. If $\mu_x$ denotes the expected time to reach $x^*$ from the left, then you have the following recurrence equation

$$ \mu_x=1+[p_x\mu_x+(1-p_x)\mu_{x+1}] $$ and obviously $\mu_{x^*}=0$. This yields $$ \mu_x=\mu_{x+1}+\frac1{1-p_x}. $$ Hence $$ \mu_x=\frac1{1-p_x}+\frac1{1-p_{x+1}}+\dots+\frac1{1-p_{x^*-1}}. $$

  • 1
    $\begingroup$ Thanks for the answer, I think I was approaching the problem from another direction, and I was not seeing this simplification, which makes things much clearer. $\endgroup$ Commented Nov 25, 2023 at 11:12

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .