Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Having a defined markov chain with a known transition matrix, rather than to calculate the steady state probabilities, I would like to simulate and estimate them.

Firstly, from my understanding there is a transient/warm-up time we have to cater for, therefore run our markov chain for sufficiently long time in order to BE in a steady state. Any ideas how to estimate that?

Secondly, even if we discard the initial k observations, how do we actually measure the probabilities? I have the following two approaches I would use:

  1. after the initial transient period observe which state we are and register that. Rerun the same simulation and register then which state we are. aftern N runs, take average of each occurences of each state to get approximations. Problems: Bloody too inefficient

  2. After the initial transient time, generate N state transitions and count occurences (WITHOUT RESTARTING). Take average. Problems: Each of N samples are not independent

I was wondering whether you guys could point me into the right direction and spot any flaws in my logic. Thank you in advance for your help.

share|cite|improve this question

If you know that your chain is ergodic then the second approach is the natural and efficient way.

share|cite|improve this answer

I guess both techniques are OK, it's just the questions of how "accurate" you want your results to be - tradeoff between speed and accuracy.

share|cite|improve this answer

Your first question is essentially "How long does it take for a Markov chain to get close to its stationary distribution?" Obviously the answer depends on the structure of the chain in question, and it isn't always easy to estimate just by looking at the chain. There is a lot of research on this question under the name of "mixing times".

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.