Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.
  1. In a continuous-time Markov chain, I was wondering why the holding time and the next state are independent? Are the independence a conditional one given the current state?

  2. Quoted from Ross's Stochastic processes:

    The amount of time the process spends in state $i$, and the next state visited, must be independent random variables. For if the next state visited were dependent on $\tau_i$, then information as to how long the process has already been in state $i$ would be relevant to the prediction of the next state—and this would contradict the Markovian assumption.

    One can also find identical claim at another book here with more
    context available.

    I don't understand why if the two are dependent, the Markov property is violated.

Thanks and regards!

share|improve this question

2 Answers 2

up vote 1 down vote accepted

For question 1, the answer is yes. You might first consider the case of 2 possible "next states", which corresponds to showing that if $U,V$ are two independent exponential random variables (with possibly different rates), then $\min(U,V)$ is independent of the event $U < V$. Counterintuitive but true.

For question 2, consider a simple example: suppose $X_t$ is a process that, whenever it enters a certain state $a$, it does one of two things: waits 5 seconds and then transitions to $b$, or waits 10 seconds and then transitions to $c$. Here the holding time is completely correlated with the next state. Such a process cannot be Markov. For instance, we have $P(X_{20} = b | X_{17} = a) > 0$, but $P(X_{20} = b | X_{17} = a, X_{14} \ne a) = 0$.

Essentially, information about the holding time is information about the history of the process (where it was at an earlier time), and in a Markov process this is not allowed to prejudice where it goes next.

share|improve this answer
    
Thanks! (1) I was wondering why given the current state, it corresponds to independent U and V? (2) When U and V are independent exponential r.v.s, why min(U,V) is independent of the event U<V? (3) In the example, at state a, the intervals in the two possibilities are 5 and 10 seconds, while in $P(X_{20} = b | X_{17} = a) $ and $P(X_{20} = b | X_{17} = a)$, the intervals are 3. How do you know the $P(X_{20} = b | X_{17} = a) > 0$ and $P(X_{20} = b | X_{17} = a) > 0$? –  Tim May 9 '11 at 5:24

The next state depends on the recent states, not how long you stayed in them. Notice that this is implicit in the state transition matrix.

share|improve this answer
    
Thanks! Could you elaborate on "this is implicit in the state transition matrix"? –  Tim May 2 '11 at 0:19
    
There is no information in the state transition matrix about holding times. It merely tells you the distribution of the next state given the past states. –  Emre May 2 '11 at 0:21

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.