2
$\begingroup$

I have been trying to learn more about different types of Markov Chains. So far, here is my basic understanding of them:

  • Discrete Time Markov Chain: Characterized by a constant transition probability matrix "P"
  • Continuous Time Markov Chain: Characterized by a time dependent transition probability matrix "P(t)" and a constant infinitesimal generator matrix "Q". The Continuous Time Markov Chain is based on the Exponential Distribution and thereby must obey the Memoryless Property.
  • Non Homogenous Continuous Time Markov Chain: Characterized by a time dependent transition probability matrix "P(t)" and a time dependent infinitesimal generator matrix "Q(t)". Non Homogenous Continuous Time Markov Chains are not necessarily based on the Exponential Distribution and thereby do not need to obey the Memoryless Property.

This brings me to my question - given the above information, I am having difficulty understanding the difference between these Markov Chains and a Semi-Markov Process.

Based on some readings that I have done, it seems like a Semi Markov Process is characterized by its "Sojourn Time" - that is, the amount of time that is spent in some state. Since the probability distribution of this "Sojourn Time" does not necessarily need to be Exponentially Distributed, a Semi Markov Process does not need to obey the Memoryless Property. If this is the case, then how exactly is the Semi-Markov Process different from a Non Homogenous Continuous Time Markov Chain?

Can someone please help me understand this? Perhaps some example could illustrate in which conditions it might be more advantageous to use a Semi-Markov Process compared to a Continuous Markov Process and vice versa?

Thanks!

Note: I am currently reading this reference : Difference between non-homogeneous Markov and Semi-Markov?

Note: I understand that as the name suggests, "Semi-Markov" does not need to obey the "Markov Property". That is, in Semi-Markov, the future states that the process can go to does not only depend on the current state, and can also depend on the history of the process. This is in contrast to a Continuous Markov Process in which the Markov Property must be obeyed, and the future state of a Continuous Markov Process can only depend on the current state.

Note: I am told that one of the differences between Semi-Markov and Markov is how the notion of time is recorded. In a Continuous Markov Process, time is recorded in the "clock forward" format - this means that all time transitions are recorded from some initial time. In a Semi Markov Process, time is recorded in the "clock reset" format - this means that the clock is restarted every time a transition is recorded. Supposedly, "clock forward" is not able to take into consideration the history of the Markov Process - whereas "clock reset" is able to take into consideration the history of the Markov Process. However, I do not understand this point - I am not sure as to why recording time using different formats "magically" allows the Markov Process to consider the history or not to consider the history of the process.

$\endgroup$

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.