# Independence of holding time and next state in continuous-time Markov chain

1. In a continuous-time Markov chain, I was wondering why the holding time and the next state are independent? Are the independence a conditional one given the current state?

2. Quoted from Ross's Stochastic processes:

The amount of time the process spends in state $i$, and the next state visited, must be independent random variables. For if the next state visited were dependent on $\tau_i$, then information as to how long the process has already been in state $i$ would be relevant to the prediction of the next state—and this would contradict the Markovian assumption.

One can also find identical claim at another book here with more
context available.

I don't understand why if the two are dependent, the Markov property is violated.

Thanks and regards!

-

For question 1, the answer is yes. You might first consider the case of 2 possible "next states", which corresponds to showing that if $U,V$ are two independent exponential random variables (with possibly different rates), then $\min(U,V)$ is independent of the event $U < V$. Counterintuitive but true.

For question 2, consider a simple example: suppose $X_t$ is a process that, whenever it enters a certain state $a$, it does one of two things: waits 5 seconds and then transitions to $b$, or waits 10 seconds and then transitions to $c$. Here the holding time is completely correlated with the next state. Such a process cannot be Markov. For instance, we have $P(X_{20} = b | X_{17} = a) > 0$, but $P(X_{20} = b | X_{17} = a, X_{14} \ne a) = 0$.

Essentially, information about the holding time is information about the history of the process (where it was at an earlier time), and in a Markov process this is not allowed to prejudice where it goes next.

-
Thanks! (1) I was wondering why given the current state, it corresponds to independent U and V? (2) When U and V are independent exponential r.v.s, why min(U,V) is independent of the event U<V? (3) In the example, at state a, the intervals in the two possibilities are 5 and 10 seconds, while in $P(X_{20} = b | X_{17} = a)$ and $P(X_{20} = b | X_{17} = a)$, the intervals are 3. How do you know the $P(X_{20} = b | X_{17} = a) > 0$ and $P(X_{20} = b | X_{17} = a) > 0$? – Tim May 9 '11 at 5:24

The next state depends on the recent states, not how long you stayed in them. Notice that this is implicit in the state transition matrix.

-
Thanks! Could you elaborate on "this is implicit in the state transition matrix"? – Tim May 2 '11 at 0:19
There is no information in the state transition matrix about holding times. It merely tells you the distribution of the next state given the past states. – Emre May 2 '11 at 0:21