# Calculating the expected hitting time of a specific birth and death chain

I am working with a specific birth and death chain, defined as follows.

Consider a set of states $$X = \{0,1,2,...,n\}$$, where $$x^* \in (0,n)$$ is a recurrent state. Transition probabilities are defined as follows:

$$P_{i,j} = \left\{\begin{array}{lc} 1-p_i & \text{ if } i = j+1 \\ p_i & \text{ if } i = j \\ 0 & \text{ otherwise}\end{array}\right., \forall i,j \neq \{0,x^*,n\} \in X.$$

Given that $$x^*$$ is a recurrent state, $$P_{x^*,x^*}=1$$. Moreover, $$P_{0,0}=0$$, and also $$P_{n,n}=0$$.

I am calculating the expected hitting time starting from any state $$x < x^*$$ (i.e., the hitting time from below). I call this value $$\eta_{x,x^*}$$ in line with standard notation on Markov Chains.

Starting from the constraint that $$\eta_{x^*,x^*} =0$$, I retrieved $$\eta_{x^*-1,x^*} = \frac{1}{1-P_{x^*-1,x^*-1}}$$, and then

$$\eta_{x^*2,x^*} = \frac{1}{1-P_{x^*-2,x^*-2}} + \frac{P_{x^*-2,x^*-1}}{(1-P_{x^*-1,x^*-1})(1-P_{x^*-2,x^*-2})},$$

$$\eta_{x^*-3,x^*} = \frac{1}{1-P_{x^*-3,x^*-3}} \left(1 + \frac{P_{x^*-3,x^*-2}}{1-P_{x^*-2,x^*-2}} + \frac{P_{x^*-3,x^*-2}P_{x^*-2,x^*-1}}{(1-P_{x^*-1,x^*-1})(1-P_{x^*-2,x^*-2})}\right).$$

Proceeding backward, I came up with this general formula for any state $$x < x^*$$:

$$\eta_{x,x^*} = \frac{1}{1-P_{x,x}} \left(1 + \sum_{i=x}^{x^*-2}\prod_{l=x}^i\frac{P_{l,l+1}}{1- P_{l+1,l+1}} \right),$$

Can someone tell me if this formula is correct?

Thank you in advance, I am happy to give more details in case they are needed.

• Note that I just deleted the second part of the question which is more problematic, and so, I could ask it in a second question Commented Nov 20, 2023 at 16:11

Your Markov chain has the property that from $$i$$ you either go to $$i+1$$ or just stay there. If $$\mu_x$$ denotes the expected time to reach $$x^*$$ from the left, then you have the following recurrence equation

$$\mu_x=1+[p_x\mu_x+(1-p_x)\mu_{x+1}]$$ and obviously $$\mu_{x^*}=0$$. This yields $$\mu_x=\mu_{x+1}+\frac1{1-p_x}.$$ Hence $$\mu_x=\frac1{1-p_x}+\frac1{1-p_{x+1}}+\dots+\frac1{1-p_{x^*-1}}.$$

• Thanks for the answer, I think I was approaching the problem from another direction, and I was not seeing this simplification, which makes things much clearer. Commented Nov 25, 2023 at 11:12