# Continuous-time Markov Chain question

Consider an immigration-death model $X = (X_t)_{t\geq0}$, i.e. a model where immigrants arrive according to a Poisson process with rate $\lambda$ and individuals have independent $Exp(\mu)$ lifetimes.

Suppose $\lambda > 0$. Suppose $X_0 = 0$.

(i) Perform a one-step analysis to justify carefully that $\Phi(t) = \mathbb{E}[z^{X_t} ]$ satisfies $$\Phi(t)=e^{-\lambda t}+\int_0^t\lambda e^{-\lambda s}\Phi(t-s)(1-(1-z)e^{-\mu(t-s)})ds$$

(ii) Solve the integral equation to determine the distribution of $X_t$.

This is an exam question but I can't seem to wrap my head around it:

My attempt: $\mathbb{E}[z^{X_t}]=\int_0^{\infty}\mathbb{E}[z^{X_t}|T_1=s]\lambda e^{-\lambda s}ds=\int_0^t\mathbb{E}[z^{X_t}|T_1=s]\lambda e^{-\lambda s}ds+\int_t^{\infty}\mathbb{E}[z^{X_t}|T_1=s]\lambda e^{-\lambda s}ds=\int_0^t\mathbb{E}[z^{X_t}|T_1=s]\lambda e^{-\lambda s}ds+e^{-\lambda t}$

Now my idea is to apply the Markov property at the time $T_1=s$ which is a stopping time. However, I can't finish the proof since I don't get the extra multiplier $(1-(1-z)e^{-\mu(t-s)})ds$

For (ii) I did a standard multiplication and differentiation to get that the following holds:

$$\Phi'(t)=-\lambda(1-z)e^{-\mu t}\Phi(t)$$

which after integrating gives

$$\Phi(t)=Ce^{\frac{\lambda(1-z)}{\mu}e^{-\mu t}} \text{ for } \mu\neq 0$$

and

$$\Phi(t)=Ce^{-\lambda(1-z)t} \text{ for } \mu=0$$

I can't associate this with any distribution and furthermore, I have a feeling I've messed up the calculations since $\Phi(0)$ should equal $0$. Any help is appreciated!

This looks like an analysis of the transient behavior of an $M/M/\infty$ queue.

1) The formula for $\phi(t)$ you are working towards looks correct. Your work also looks correct (here I assume $T_1$ is the time of the first arrival, and the system is initially empty). The extra step is to compute $$E[z^{X_t}|T_1=s] = E[z^Az^{X_{t-s}}|T_1=s]$$ where $A$ is a Bernoulli random variable that is 1 if the arrival at time $s$ has not yet left by time $t$, and $0$ else.

2) Note that $\phi(0) = E[z^0]=1$. Let's see, I find the change of variables $x=t-s$ to be useful: \begin{align} x &= t-s\\ dx &= -ds \end{align} $$\phi(t) = e^{-\lambda t} + \int_0^t \lambda e^{-\lambda (t-x)} \phi(x)[1-(1-z)e^{-\mu x}]dx$$ and multiplying by $e^{\lambda t}$ gives $$\phi(t) e^{\lambda t} = 1 + \int_0^t \lambda e^{\lambda x} \phi(x) [1-(1-z)e^{-\mu x}]dx$$ which is a nice form for differentiation. Okay this gives the same solution that you get: $$\phi'(t) = -\lambda (1-z)e^{-\mu t} \phi(t)$$ Assuming $\mu>0$ and using separation of variables, I get the same solution as you $$\phi(t) = C e^{(\lambda/\mu)(1-z)e^{-\mu t}}$$ So it remains only to solve for $C$ (which can depend on $\lambda, \mu, z$) by using $\phi(0)=1$ (not $\phi(0)=0$). You can test your final answer by letting $t\rightarrow\infty$, so $\phi(t)$ should converge to the z-transform of the Poisson distribution.

• Can you elaborate more on the first step, please? – asdf Apr 29 '18 at 15:32