# Please can someone help me to understand stationary distributions of Markov Chains?

I'm currently trying to understand (intuitively) what a stationary distribution of a Markov Chain is? In our lecture notes, we're given the following definition:

This was of little benefit to my understanding, so I've tried searching online for a more useful explanation. I then found the following video, which improved my understanding to the extent that I now understand that stationary distributions are to do with looking at what happens to the probabilities at each state within a Markov Chain when time becomes infinitely large. This is still not a sufficient enough understanding of the concept though.

For example, I've been asked to show that $$\pi_{a} = \left( \frac{2}{5}, \frac{3}{5}, 0, 0, 0 \right) \\ \pi_{b} = \left( 0, 0, 1, 0, 0 \right) \\ \pi_{c} = \left( 0, 0, 0, \frac{3}{5}, \frac{2}{5} \right)$$ are stationary distributions with respect to the Markov Chain with one-step transition martix $$\mathbf{P} = \left( \begin{array}{ccccc} \frac{1}{2} & \frac{1}{2} & 0 & 0 & 0 \\ \frac{1}{3} & \frac{2}{3} & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & \frac{2}{3} & \frac{1}{3} \\ 0 & 0 & 0 & \frac{1}{2} & \frac{1}{2} \end{array} \right)$$

How would you do this? What is a stationary distribution, with respect to this example?

Also, could someone please confirm that I'm correct in thinking that the notation $p_{ij}$ denotes the probability of the process moving from the state $i$ to the state $j$?

• To answer the problem they gave you, you need to show that each of $\pi_a,\pi_b, \pi_c$ satisfy all three conditions $S1,S2,S3$ to be a stationary distribution. Mar 7, 2016 at 0:08
• A stationary distribution is one which, if your system starts with that distribution, it stays at that distribution. It is closely related to, but not the same as, a limiting distribution. More precisely, there is a theorem which says that a certain class of Markov chains have a unique stationary distribution and always converge to this stationary distribution. But other chains have a unique stationary distribution and don't approach it, while still others approach one of their non-unique stationary distributions.
– Ian
Mar 7, 2016 at 0:15
• @Ian where can I find what class of chains belong to the different possible situations you have described? Nov 8, 2016 at 18:46
• @atulgangwar I don't really know a name for the result but irreducible chains have a unique stationary distribution while aperiodic chains approach some stationary distribution. (Both of these assume a finite state space.)
– Ian
Nov 8, 2016 at 18:50

"Also, could someone please confirm that I'm correct in thinking that the notation $p_{ij}$ denotes the probability of the process moving from the state $i$ to the state $j$?" $(*)$

Correct

"How would you do this? What is a stationary distribution, with respect to this example?"

If the chain starts in state $3$ it stays there forever because according to $(*)$ there is zero probability to move to another state.

Therefore

$\pi_{b} = ( 0, 0, 1, 0, 0)$

is an obvious stationary distribution.

If the chain starts in state $1$ or $2$ it stays there forever because according to $(*)$ there is zero probability to move to another state.

If the chain starts in state $4$ or $5$ it stays there forever because according to $(*)$ there is zero probability to move to another state.

Now you can treat these as two $2 \times 2$ matrices and use the result that a vector which fulfills:

$\mathbf{\hat{\pi}} \mathbf{P} = \mathbb{\hat{\pi}}$ $\:\:(**)$

is a stationary distribution.

So you solve these two sets of systems of equations to get the remaining stationary distributions. Here you also need to use that $\hat{\pi}$ is a probability vector; that is, its components sum to one.

"I now understand that stationary distributions are to do with looking at what happens to the probabilities at each state within a Markov Chain when time becomes infinitely large"

You also have this theorem that can be good to know:

If the Markov chain is irreducible and aperiodic then

$\lim \limits_{n \to \infty} P^n = \hat{P}$

where $\hat{P}$ is a matrix whose rows are identical and equal to the stationary distribution $\mathbb{\hat{\pi}}$ for the Markov chain defined by equation $(**)$.

• Thanks for the in depth and helpful answer. Apologies for the delay in acknowledging it. Mar 7, 2016 at 17:29