# Birth and death process-Markov chain -continuous time

I need help to know if the result obtained from the following problem is correct, or if there is a better way to solve it.

Clients arrive at a bank according to a Poisson process of parameter lambda> 0 to a waiting system with a single server. The customer-to-customer services consist of two independent and suc- cessive stages, each lasting a random time that is distributed according to an exponential law of mu> 0 parameter. The length of service in each stage are independent of each other and of the arrivals to the system. The same server attends both stages so it can not receive a second person in stage one when the person who proceeds is in stage two. Write the Q generator of a Markov chain in continuous time that model the number of clients in the system, and explain what the states of the chain represent. Determine under what conditions there is an invariant vector for Q and have such a vector.

The form in which I determine my solution is the following.

First, I construct a process $Y_t$ that represents the number of cuts left to the process to finish all the services of all the clients that are in the system. If there are n clients in the system, then the number of stages you have to complete to complete all services are $2(n-1)-1$.

The generator Q associated with this process has the form

$$Q=\left(\begin{matrix} -\lambda& 0& \lambda& 0& \cdots\\ \mu& -\lambda -\mu& 0& \lambda& \\ 0& \mu& -\lambda-\mu& 0&\cdots\\ 0& 0& \mu& -\lambda-\mu& \\ \vdots& & \vdots& & \ddots \end{matrix}\right)$$ Using the generator Q and the relation that fulfills an invariant measure, and that is $\pi Q=0$, we have

\begin{aligned} \lambda\pi_0&=\mu\pi_1\\ (\lambda+\mu)\pi_1&=\mu\pi_2\\ (\lambda+\mu)\pi_n&=\lambda\pi_{n-1}+\mu\pi_{n+1},\quad n=2,3,\cdots, \end{aligned} If the relationship is considered time, $\pi_n=x^n$, then it will have

$$(\lambda+\mu)x^n=\lambda x^{n-2}+\mu x^{n+1}$$

Dividing by $x^{n-2}$ is achieved

$$(\lambda+\mu)x^2=\lambda+\mu x^2$$

Such a polynomial has roots

\begin{aligned} x&=1\\ x_1&=\frac{1}{2}(\rho+\sqrt{\rho^2+4\rho})\\ x_2&=\frac{1}{2}(\rho-\sqrt{\rho^2+4\rho}), \quad when \quad \rho=\frac{\lambda}{\mu} \end{aligned}

It can be shown that any combination of the following form is satisfied

$$\pi_n=c_1x_1^n+c_2x_2^n, \quad n=0,1, \cdots,$$ It can also show that using the above relation with two equations, we have a unique colution for the coefficients $c_k$, in the form

$$c_k=\frac{1-2\rho}{\prod_{j\neq k}(1-x_j/x_k)}$$ Concluding that the invariant distribution of the Y-chain has elements of the form

$$\pi_n=\frac{1-2\rho}{\sqrt{\rho^2+4\rho}}\left\{\left[\frac{1}{2}\left(\rho+\sqrt{\rho^2+4\rho}\right)\right]^n-\left[\frac{1}{2}\left(\rho-\sqrt{\rho^2+4\rho}\right)\right]^n\right\}, \quad n\geq 0$$

If I now define the process (which is really what interests me) $X_t$, as the process that counts in the number of clients in the system at time $t$, then using the above I was able to see that the invariant distribution For this process has elements of the form

\begin{aligned} \gamma_n&=\sum_{k=2(n-1)+1}^{2n}\pi_k\\ &=\sum_{j=1}^2\sum_{k=2(n-1)+1}^{2n}c_jx_j^k\\ &=\sum_{j=1}^2c_j(x_j^{-1}+1)x_j^{2n} \end{aligned} That is, the mixture of two geometric distributions.

And finally it can be seen that the condition for the $\gamma$ invariant measure to exist is that $x_1 <1$ and $x_2 <1$, or equivalently that 2 $\lambda <\mu$.

• I also want to comment that among my colleagues we have discussed a lot in how to define the states of the process and as a consequence of this have come various solutions and interpretations. – Joda Mar 28 '17 at 8:28

The method seems fine. The process you have is known as a $M/E_2/1$ queue, where $E_2$ denotes the Erlang-2 distribution. The more general $M/G/1$ queue, where the service rates follow some general distribution, is very well-studied.
The $M/E_r/1$ queue is analyzed in detail here: http://www.win.tue.nl/~iadan/queueing.pdf. You should find that the method used is very similar to yours.