What does the steady state represent to a Markov Chain?

Question

I'm a little confused as to the interpretation of the steady state in the context of a Markov chain. I know Markov chains are memoryless, in that each state only depends on its immediate predecessor, but doesn't that mean the system is in a sort of steady state?

Answer 1

One question we can ask is for the probabilities that the system will be found in each of the states at a given future time. In general, these probabilities will depend on which future time you are asking about. In many, but not all, Markov chains, however, the probability for a particular one of the states will approach a limiting value as time goes to infinity. In other words, in the far future, the probabilities won't be changing much from one transition to the next. These limiting values are called "stable probabilities".

If start off the system so that each state has probability equal to its stable probability, then these probabilities will persist for all time. The system will therefore be in a "steady state".

Some Markov chains do not have stable probabilities. For example, if the transition probabilities are given by the matrix $$ \begin{bmatrix}0 & 1\\1 & 0\end{bmatrix}, $$ and if the system is started off in State 1, then the probability of finding the system in State 1 will oscillate between 0 and 1 forever.