# Example of a continuous Markov renewal process

Clients arrive at 1 server according to a Poisson process with intensity $a$ and leave according to a Poisson process with intensity $b$. However if there are already $N$ clients waiting, a new client will also wait with probability $1/(N+1)$ or leave with probability $N/(N+1)$.

1. How do I show that the stationary distribution of the number of clients present is Poisson distributed with expectation $a/b$?
2. What is the average number of clients that arrive per unit time?

What I know:
Our state space is $\mathbb{N}_0$, the number of clients present. I thought that a state $i$ can only transition to $i-1,i,i+1$ with intensity $b,-(a+b),a$ resp. for $i<N$. For $i\geq N$ the intensity to $i+1$ is $a/(N=1)$, to $i$ is $-a/(N+1)-b$ and to $i-1$ remains $b$. These are the entries of the transition rate matrix $G$.
The transition rate matrix G is given by $g_{ij} = \left\{ \begin{array}{lr} v_ip_{ij} & : i \neq j\\ -v_i & : i=j \end{array} \right.$

where the residence time in state $i$ is distributed exponentially with parameter $v_i$ and $p_{ij}$ are the transition probabilities.
Further the stationary distribution is $P=(P_j)_j$ where $P_j=\lim_{t\rightarrow\infty}p_{ij}(t)$. And $P$ is the solution of $v_jx_j=\sum_{k\neq j}g_{kj}x_k$.

How can I use these to find 1. and 2.?

I suppose that $N$ is not a fixed number, but for all $N\in\mathbb{N}=\left\{0,1,2,\cdots\right\}$. In other words, whenever there are $N$ clients at the server, a new arriving client would choose to wait with probability $1/\left(N+1\right)$ and to leave with probability $N/\left(N+1\right)$.

• Let $N_t$ be the total number of clients at the server at the moment $t$.
• Let $A_t$ be the total number of arriving clients by the moment $t$, with $$A_t\sim\text{Poisson}(at).$$
• Let $\xi_t$ be the wait-or-leave choice of an arriving client at the moment $t$, with $$\xi_t|N_t\sim\text{Bernoulli}\biggl(\frac{1}{N_t+1}\biggr).$$
• Let $L_t$ be the total number of leaving clients by the moment $t$, with $$L_t\sim\text{Poisson}(bt).$$
• Let $A_t$, $L_t$ and $\xi_t|N_t$ be independent processes.

With these settings, the model reads $${\rm d}N_t=\xi_t{\rm d}A_t-{\rm d}L_t,$$ or in the infinitesimal increment form, $$N_{t+{\rm d}t}-N_t=\xi_t\left(A_{t+{\rm d}t}-A_t\right)-\left(L_{t+{\rm d}t}-L_t\right).$$

Denote $$p_n(t)=\mathbb{P}\left(N_t=n\right).$$ We have \begin{align} p_n(t+{\rm d}t)&=\mathbb{P}\left(N_{t+{\rm d}t}=n\right)\\ &=\mathbb{P}\left(N_{t+{\rm d}t}=n|N_t=n-1\right)\mathbb{P}\left(N_t=n-1\right)\\ &\quad+\mathbb{P}\left(N_{t+{\rm d}t}=n|N_t=n\right)\mathbb{P}\left(N_t=n\right)\\ &\quad+\mathbb{P}\left(N_{t+{\rm d}t}=n|N_t=n+1\right)\mathbb{P}\left(N_t=n+1\right). \end{align} This relation eventually lead us to \begin{align} p_0'(t)&=-ap_0(t)+bp_1(t),&&n=0,\\ p_n'(t)&=\frac{a}{n}p_{n-1}(t)-\left(\frac{a}{n+1}+b\right)p_n(t)+bp_{n+1}(t),&&n\ge 1, \end{align} where $$p_n'(t)=\lim_{{\rm d}t\to 0^+}\frac{p_n(t+{\rm d}t)-p_n(t)}{{\rm d}t}.$$

As for stationary distribution, we have $p_n'(t)=0$. Thus \begin{align} -ap_0+bp_1&=0,&&n=0,\\ \frac{a}{n}p_{n-1}-\left(\frac{a}{n+1}+b\right)p_n+bp_{n+1}&=0,&&n\ge 1. \end{align} In addition, the conservation of probability requires $$\sum_{n=0}^{\infty}p_n=1.$$

The above iterative scheme can be solved inductively. Note that the first three terms read \begin{align} p_1&=\frac{a}{b}p_0,\\ p_2&=\frac{1}{2}\left(\frac{a}{b}\right)^2p_0,\\ p_3&=\frac{1}{6}\left(\frac{a}{b}\right)^3p_0. \end{align} These inspire to show inductively that $$p_n=\frac{1}{n!}\left(\frac{a}{b}\right)^np_0$$ for all $n\in\mathbb{N}$. Finally, the conservation of probability yields $$1=\sum_{n=0}^{\infty}p_n=p_0\sum_{n=0}^{\infty}\frac{1}{n!}\left(\frac{a}{b}\right)^n=p_0\exp\left(\frac{a}{b}\right)\iff p_0=\exp\left(-\frac{a}{b}\right).$$ Consequently, $$p_n=\frac{1}{n!}\left(\frac{a}{b}\right)^n\exp\left(-\frac{a}{b}\right)$$ for all $n\in\mathbb{N}$.

Obviously, this is a Poisson distribution with parameter $a/b$.

As per the average number of arriving clients per unit time, it should be $$\mathbb{E}\left(A_1\right)=a.$$