2
$\begingroup$

Given a discrete-time Markov chain without independent increments, is the embedding of it into a continuous time Markov chain (i.e. via the use of exponential waiting times) an example of a continuous time Markov process without independent increments?

Note: This question is forked from one I asked previously, since the Ornstein-Uhlenbeck process was an answer to every other part of that question except the part being asked again here.

Context:

As an answer to this related question, user @madprob gave an example of a discrete time Markov chain that does not have independent increments.

Specifically, let a discrete time process with state space $\mathbb{R}$ be defined as follows: $X_{n+1} = X_n + Z$, where $Z|(X_n,X_{n-1},...,X_0) \sim N(-X_n, 1)$.

In general, any Markov process in discrete time can be written as $X_{n+1}=X_n+Z_{n+1}$ where $Z_n = f(X_n,U_n)$ for some suitable $f$ and a random variable $U_n$ that is independent of $(X_{n-1},...,X_1,X_0)$ -- thus the problem of finding a discrete time Markov process that does not have independent increments reduces to the problem of choosing an appropriate $f$.

Such a counterexample presumably exists given the answer to this question.

EDIT: I suppose the process of embedding a Markov chain into continuous time is essentially subordinating a Poisson process. So this question is probably just a special case of whether or not subordinating infinitely divisible distributions preserves independence properties.

$\endgroup$
2
$\begingroup$

This isn't an answer per se, rather some thoughts to consider.

If $\{X_n:n=1,2,\ldots\}$ is a Markov chain and we define the holding times $\{T_n:n=1,2,\ldots\}$ by $T_n\sim\mathsf{Exp}(\lambda_{X_n})$, then the sequence $\{T_n\}$ only defines a Poisson process if $\lambda_{X_n}$ is equal for all $n$. Otherwise, the intensity would be a stochastic process itself, in which case the counting process $N(t):= \sum_{n=1}^\infty \mathsf 1_{(0,t]} T_n$ would be a Cox process. (The jump process defined by the state transitions does of course define a CTMC, though.)

If the embedded chain can have same-state transitions with positive probability, then a jump time would be be a geometric sum of the holding time for the current state and therefore exponentially distributed, so the jump process would still be a CTMC.

I suspect there may be a clever way of defining the holding times so that the CTMC would have independent increments even if the embedded chain did not, but am not sure yet whether that is the case.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.