1
$\begingroup$

What's the difference between stationary distribution and limiting distribution of a finite state markov chain? Do stationary distributions always exist and limiting distributions may not necessarily exist?

What I know so far:

A stationary distribution $\pi_\infty$ satisfies $\pi_\infty = P^T \pi_\infty$ and that limiting distributions are a subset of stationary distributions.

$\endgroup$
1
$\begingroup$

A Markov chain $\{X_n: n=0,1,2,\ldots\}$ on $\{0,1,\ldots,N\}$ with transition matrix $P$ is ergodic if it is aperiodic and positive recurrent. (Note that positive recurrence is equivalent to recurrence in the finite-state case.) In this case a unique stationary distribution $\pi$ exists satisfying $\pi = \pi P$ (here we are considering $\pi$ as a row vector). Moreover, $\pi$ is also a limiting distribution, in that $$\lim_{n\to\infty} \mathbb P(X_n=j\mid X_0=i) = \pi_j $$ for any $i\in\{0,1,\ldots,N\}$.

If we remove the requirement of aperiodicity then $X$ can have a stationary distribution $\pi$ that is not a limiting distribution; indeed $X$ need not have a limiting distribution at all. For example, consider the transition matrix $$ P = \begin{bmatrix}0&1\\1&0\end{bmatrix}. $$ Then if we take $\pi = \begin{bmatrix}\frac12&\frac12\end{bmatrix}$, we have $\pi P=\pi$, so that $\pi$ is a stationary distribution for $P$, but $$ P^n = \begin{cases} \begin{bmatrix}0&1\\1&0\end{bmatrix},& n \text{ even}\\ \begin{bmatrix}1&0\\0&1\end{bmatrix},& n \text{ odd}, \end{cases} $$ so that $\lim_{n\to\infty}\mathbb P(X_n=j\mid X_0=i)$ does not exist for $i=0$ or $i=1$.

If we remove the condition of recurrence, then $X$ may have a limiting distribution conditional on the initial state, and even infinitely many stationary distributions. Consider the transition matrix $$ P = \begin{bmatrix}1&0\\0&1\end{bmatrix}. $$ Then $\pi P = \pi$ for any $\pi = \begin{bmatrix}\pi_0&\pi_1\end{bmatrix}$ with $\pi_0,\pi_1\geqslant 0$ and $\pi_0+\pi_1=1$, so that any distribution is a stationary distribution for $P$. Moreover, $$ \lim_{n\to\infty} \mathbb P(X_n = j\mid X_0 = i) = \begin{cases}1,& i=j\\ 0,& i\ne j.\end{cases} $$

Every finite-state Markov chain has at least one stationary distribution $\pi$, even if it is periodic and transient. In order to find Markov chains without stationary distributions, we must look to the countably infinite state space. If you are interested in that case, let me know and I can expand this answer.

$\endgroup$
2
  • $\begingroup$ Could you provide a reference to a proof of the statement "Every finite-state Markov chain has at least one stationary distribution 𝜋, even if it is periodic and transient"? I feel I keep seeing this statement, but can never actually find a proof written down. Apologies for any stupidity on my part inherent in the question. $\endgroup$ Oct 4 '20 at 23:21
  • 1
    $\begingroup$ @Raj It is a consequence of the Perron-Frobenius theorem. $\endgroup$
    – Math1000
    Oct 5 '20 at 1:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.