Two books construct Markov processes from Q-matrices using waiting times and jump chains but differ in whether the waiting times depend on the current state. Can the two be reconciled?

Klenke claims that to every Q-matrix $q$ corresponds a unique Markov process that is differentiable at time $t=0$ and such that $q$ is its derivative there (Theorem 17.25). According to him, this unique process can be constructed by combining a (discrete time) Markov chain representing "jumps" between states, and a Poisson process representing the time lapse between jumps (a.k.a. the waiting time). According to this model, the waiting time $\sim \exp(\lambda)$, where $\lambda$ is a function of $Q$, and therefore unaffected by the current state.

Norris too constructs a Markov process from a Q-matrix (section 2.6). His construction is similar to Klenke's except that $\lambda$ is a function not only of $Q$, but of the state active at the beginning of the waiting time.

Can the two constructions be reconciled? What am i missing here? Is one of the authors wrong or idiosyncratic in his definition of a Markov process?


Theorem 17.25 Let $q$ be an $E\times E$ matrix ($E$ being a countable set) such that $q(x,y)\geq0$ for all $x,y\in E$ with $x\neq y$. Assume that the following hold

i) $q(x,y)\geq0$ for all $x,y\in E$,

ii) $q(x,x)=-\sum_{y\neq x}q(x,y)$

iii) $\sup_{x\in E}|q(x,x)|<\infty$

Then $Q$ is the $Q$-matrix of a unique Markov process $X$.

Proof [Abridged] Let $I$ be the unit matrix on $E$. Define $$p(x,y)=\frac{1}{\lambda}q(x,y)+I(x,y)\space\space\mathrm{for\, }x,y\in E$$

Then $p$ is a stochastic matrix and $q=\lambda(p-I)$. Let $\left((Y_n)_{n\in\mathbb{N}_0}, (\mathrm{P}_x^Y)_{x\in E}\right)$ be a discrete Markov chain with transition matrix $p$ and let $\left((T_t)_{t\geq0}, (\mathrm{P}_n^T)_{n\in \mathbb{N}_0}\right)$ be a Poisson process with rate $\lambda$. Let $X_t:=Y_{T_t}$ and $\mathrm{P}_x=\mathrm{P}_x^Y\otimes\mathrm{P}_0^T$. The $\mathfrak{X}:=\left((X_t)_{t\geq0}, (\mathrm{P}_x)_{x\in E}\right)$ is [...] the required Markov process. [...] $\square$


A minimal right-continuous process $(X_t)_{t\geq0}$ on $I$ is a Markov chain with initial distribution $\lambda$ and generator matrix $Q$ if its jump chain $(Y_n)_{n\geq0}$ is discrete-time Markov($\lambda$, $\Pi$) and if for each $n\geq1$, conditional on $Y_0, \dots, Y_{n-1}$, its holding times $S_1, \dots, S_n$ are independent exponential random variables of parameters $q(Y_0), \dots, q(Y_{n-1})$ respectively. We say $(X_t)_{t\geq0}$ is Markov($\lambda$, $Q$) for short. We can construct such a process as follows: let $(Y_n)_{n\geq0}$ be discrete-time Markov($\lambda$, $\Pi$) and let $T_1,T_2,\dots $ be independent exponential random variables of parameter $1$, independent of $(Y_n)_{n\geq0}$. Set $S_n=T_n/q(Y_{n-1})$, $J_n=S_1+\cdots+S_n$ and $$X_t=\begin{cases}Y_n &\mathrm{if\, }J_n\leq t<J_{n+1}\mathrm{\, for\, some\, }n\\ \infty &\mathrm{otherwise}\end{cases}$$

Then $(X_t)_{t\geq0}$ has the required properties.


1 Answer 1


The two constructions are equivalent and their equivalence is based on the so-called thinning of Poisson processes.

Klenke starts from a homogenous Poisson process with a large rate $\lambda$. Amongst the times of this process, when at $x$, a relative proportion $1+q(x,x)/\lambda$ is used to jump from $x$ to $x$ and, for every $y\ne x$, a relative proportion $q(x,y)/\lambda$ is used to jump from $x$ to $y$. The jumps $x\to x$ have no effect, hence one is left with a proportion $q(x,y)/\lambda$ of jumps $x\to y$ amongst a global population of potential jump times with density $\lambda$, that is, the correct rate $q(x,y)$. The only condition for this construction to work is $1+q(x,x)/\lambda\geqslant0$ for every $x$, that is, $\lambda\geqslant\sup\limits_x[-q(x,x)]$, hence one can choose, as many authors do, $\lambda=\sup\limits_x[-q(x,x)]$ but any larger value of $\lambda$ will do as well.

Norris's construction might be more usual hence I will not comment on it here, except to note that $\lambda$, the initial distribution in Norris, is related in no way whatsoever to $\lambda$, the positive real number in Klenke. (My impression is that Klenke's version, more elegant, is slowly replacing the other one in the probabilists' minds.)

Edit The piece of Norris's construction missing from your account is that $\Pi$ is related to $Q$ through $\Pi(x,x)=0$ for every $x$, and, for every $y\ne x$, $$ \Pi(x,y)=\frac{q(x,y)}{q(x)}\quad\text{with}\quad q(x)=-q(x,x)=\sum_{z\ne x}q(x,z). $$


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .