Here's my understanding of it: Assume we have an $n\times n$ stochastic matrix $P$ that represents our Markov chain such that $x$ and $y$ are stationary distributions for $P$. Then

$P(x) = x$

$P(y) = y$

$P(ax+by) = P(ax) + P(by) = aP(x) + bP(y)$ where $ax+by$ is a convex linear combination

$ = ax + by$

meaning that $ax+by$ is a stationary distribution, so there is an infinite amount of stationary distributions of P if there are at least 2.

Does this mean a Markov chain either has one or infinitely many stationary distributions?

  • $\begingroup$ For a perhaps unenlightening example, let $$ P = \begin{pmatrix}1&0\\0&1\end{pmatrix}. $$ Then $\pi P = \pi$ for any row vector $\pi$, so any distribution is a stationary distribution for $P$. $\endgroup$ – Math1000 Feb 17 at 5:27

Yes. Let $\mu$ and $\nu$ be two distinct stationary distributions. Now choose randomly between $\mu$ and $\nu$ with probabilities $p$ and $1-p$, and whatever distribution is chosen, choose according to it an initial state. Then the distribution of the state at time $0$ is $p\mu+(1-p)\nu$. If we chose $\mu$, then our distribution at time $1$ is still $\mu$ (because $\mu$ is stationary), and if we chose $\nu$ our distribution at time $1$ is $\nu$. Therefore our distribution at time $1$ is still $p\mu+(1-p)\nu$, so $p\mu+(1-p)\nu$ is stationary. This is just a probabilistic proof of what you proved algebraically.


Yes, the set of stationary distributions will always be convex, for the reason you give.

Some Markov chains (like simple random walk on the integers) have no stationary distributions. But finite MCs have stationary distributions, so your statement about "1 or infinitely many" is correct, in this case.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.