Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I'm a little confused as to the interpretation of the steady state in the context of a Markov chain. I know Markov chains are memoryless, in that each state only depends on its immediate predecessor, but doesn't that mean the system is in a sort of steady state?

share|improve this question
    
Thank you @Lucius - Mods please feel free to delete both of these comments. –  leonardo Sep 11 '12 at 23:31
add comment

1 Answer 1

up vote 5 down vote accepted

One question we can ask is for the probabilities that the system will be found in each of the states at a given future time. In general, these probabilities will depend on which future time you are asking about. In many, but not all, Markov chains, however, the probability for a particular one of the states will approach a limiting value as time goes to infinity. In other words, in the far future, the probabilities won't be changing much from one transition to the next. These limiting values are called "stable probabilities".

If start off the system so that each state has probability equal to its stable probability, then these probabilities will persist for all time. The system will therefore be in a "steady state".

Some Markov chains do not have stable probabilities. For example, if the transition probabilities are given by the matrix $$ \begin{bmatrix}0 & 1\\1 & 0\end{bmatrix}, $$ and if the system is started off in State 1, then the probability of finding the system in State 1 will oscillate between 0 and 1 forever.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.