# Markov chain stationary distribution

Let $(X_n : n \in \mathbb{N}_0)$ be a Markov chain on finite state space $S = (1,2,3,4,5)$ with the transition probability matrix $P = (p_{ij})_{i,j \in S}$ satisfying

$$\sum_{i \in S}p_{ij} = 1$$

for $j \in S$

Show that $\pi = (1/5, ..., 1/5)$ is a stationary distribution of $(X_n : n \in \mathbb{N}_0)$

I know how to work out $\pi$ for a given P however I am unsure how to "create" or use the satisfying property to directly show what $\pi$ is.

Initial thoughts:

$P$ could be a diagonal matrix with entries of $1/5$ thus $\pi = (1/5, ..., 1/5)$

HINT:

The stationary distribution $\pi$ satisfies the relationship $\pi P=\pi$, which is equivalent to showing $\forall j\in S$:

$$\pi_j = \sum_{i \in S}\pi_i p_{ij}$$

You can also write out $\pi$ explicitly as a row vector where every entry is $\frac{1}{5}$, and $P$ is a matrix with entries $p_{ij}$ in row $i$, column $j$. Expanding this out and using the fact that $\sum_{i \in S} p_{ij} = 1$, you'll see very quickly how you get $\pi P = \pi$

• Could I clarify perhaps a simple notation.. Am I to assume that $\sum_{i \in S} p_{ij} = 1$ means each row equals 1? e.g $(p_{11},p_{12},p_{13},p_{14},p_{15}) = 1$ etc for $i = 1,2,3,4,5$ – Matt Nov 26 '15 at 5:14
• @Matt It means each column sums to $1$ (j is fixed so it's the column). Try writing it out with a $2x2$ matrix first to see the pattern – Brenton Nov 26 '15 at 5:17
• Ah yes, explicitly subbing in $\pi$ and expanding I get $1/5 = 1/5(p_{11},...,p_{51})$ ... $1/5 = 1/5(p_{15},...,p_{55})$. then using the relationship that $(p_{1j}+...+p_{5j}) = 1$ we can see 1/5 = 1/5 for each column. thus $\pi P = \pi$. Thank you. – Matt Nov 26 '15 at 5:31

In general, a measure $\nu:2^S\to [0,\infty)^S$ is said to be invariant under the stochastic kernel $P$ which specifies the transition probabilities of $X$, i.e. $P_{ij} = \mathbb P(X_{n+1}=j\mid X_n=i)$ for $i,j\in S$ if $$\nu = \nu P.$$ Now, since in this case we are also assuming $$\sum_{i\in S}P_{ij} = 1, \ j\in S$$ as well as the usual assumption $$\sum_{j\in S}P_{ij} =1, \ i\in S.$$ So if $\nu(i)=i$ for all $i\in S$, we have that $$\nu(j)\sum_{i\in S}P_{ij}=\nu(j),$$ and thus $\nu$ is an invariant measure for $P$.

If $|S|<\infty$ (the state space is finite, then $$\pi = \frac\nu{|S|}$$ is a stationary distribution for $P$, since $\sum_{i\in S}\pi(S) = 1$ and $\pi P=\pi$. If $S$ is countably infinite, then no stationary distribution exists.

Note in particular that in this case, the stationary distribution is uniform over the state space.