1
$\begingroup$

I study continuous time Markov chain and more specifically birth and death processes. I am trying to understand how to calculate the expectation time it takes to start from a state $i$ to a state $i+1$. I am using the book "Introduction to probability models" by Sheldon Ross but I have trouble understanding it and I did not find alternative notes on my problem so I am asking here.

We consider a general birth and death process with birth rate $\{\lambda_n\}$ and death rates $\{\mu_n\}$, where $\mu_0=0$ and we denote $T_i$ as the time it takes starting from state $i$ to enter state $i+1$. Since the times of death and births are exponential, we already know that $E[T_0]=\frac{1}{\lambda_0}$.

Then, the book follows this logic: For $i>0$, we condition whether the first transition takes the process into state $i-1$ or $i+1$, i.e we let: $$ I_i=\begin{cases} 0 \text{, if the first transition from }i\text{ is } i+1\\ 1 \text{, if the first transition from }i\text{ is } i-1 \end{cases} $$ And therefore we note that:

$E[T_i\mid I_i=1]=\frac{1}{\lambda_i+\mu_i},$

$E[T_i\mid I_i=0]=\frac{1}{\lambda_i+\mu_i}+E[T_{i-1}]+E[T_i]$

(Because independently of whether the first transition is a birth or a death, the time until it occurs is the minimum and thus ~Exp$(\lambda_i + \mu_i)$.

And here is my problem:

To compute $E[T_i]$ I simply apply the following formula: \begin{equation} E[T_i]=E[T_i\mid I_i=1]\cdot P(I_i=1)+E[T_i\mid I_i=0]\cdot P(I_i=0) \label{first} \end{equation}

and after working it out, I find:

$E[T_i]=\frac{1}{\lambda_i+\mu_i}+\frac{\mu_i}{\lambda_i(\lambda_i+\mu_i)}+\frac{\mu_i}{\lambda_i}E[T_{i-1}]$

But the book gives:

"Since the probability that the first transition is a birth is $\frac{\lambda_i}{(\lambda_i+\mu_i)}$ we have :"

$E[T_i]=\frac{1}{\lambda_i+\mu_i}+\frac{\mu_i}{(\lambda_i+\mu_i)}(E[T_{i-1}]+E[T_i])$

$\implies E[T_i]=\frac{1}{\lambda_i}+\frac{\mu_i}{\lambda_i}E[T_{i-1}]$

Obviously the use of my formula is not appropriate here and I added useless terms, but I don't see which ones and the reasoning behind it. Understanding these kind of steps would help me a lot to better grasp this topic, so any help or idea is really appreciated, thanks!

$\endgroup$
  • 2
    $\begingroup$ It happens that $$\frac{1}{\lambda_i+\mu_i}+\frac{\mu_i}{\lambda_i(\lambda_i+\mu_i)}=\frac{1}{\lambda_i}$$ hence the two recursions are equivalent. $\endgroup$ – Did Jul 16 '17 at 10:48
  • $\begingroup$ I feel a bit foolish I did not see this mini step.. Thanks for your help! $\endgroup$ – Sam Jul 16 '17 at 12:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.