Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Given a continuous Markov chains (and given the transition rates between the states) I would like to know the following:

  1. mean time of permanence for all states.
  2. higher order moments (i.e., variance and, possibly, CDF?)

In particular, I am also interested about computing the above quantities in the case when the system has to stay at least for a fixed amount of time in every state. For example, suppose there are for states {A, B, C, D}. When the system is in state A, it has to stay there for at least $T_A$ seconds, and the same for state B ($T_B$ seconds) and so on for all remaining states. In this case, is it still possible to determine the quantities in the list above, namely, mean permanence time for all states, variance and CDF?

share|cite|improve this question

If it has to stay at least a fixed amount of time in a state, it's no longer a continuous Markov chain: the future depends not only on the present but also on the past (namely how long you've been in the current state). However, if by "permanence time" you mean the amount of time it is in the state $A$ from when it enters until it first leaves that state, that will simply be $T_A + T$ where $T$ has an exponential distibution with rate $v_A$ (the rate at which the system makes transitions out of state $A$ after the initial period is up).

share|cite|improve this answer
do you know how to compute the time it takes on average to come back to a certain state when the system leaves it? – ACAC Dec 27 '11 at 16:18
Let $u_i$ be the expected time to reach state $A$ after , leaving state $i$. By a "first-step analysis", you get a system of linear equations for the $u_i$, which you can solve. – Robert Israel Dec 27 '11 at 17:59

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.