# Stationary distribution and the period of chain brand - on example

Let's define two stats Markov chain:
$P=\left( \begin{array}{ccc} 0 & 1 \\ 1/2 &1/2 \end{array} \right)$
Then, stationary distribution is:
$$sP=s$$ $$\left(0s_1+\frac12s_2, s_1+\frac12s_2\right)=\left(\frac12s_2, s_1+\frac12 s_2 \right) = (s_1, s_2)$$
And, $\frac12 s_2+ s_1+\frac12s_2 = 1$
So have exactly one solution, $s_1=\frac13, s_2=\frac23$.
Our chain has exactly one stationary distribution.

Is it ok computed ?
My mainly question is- how to compute period for state $1$ and state $2$ ?
From definition period is $o(s) =\gcd\left\{n : p_{ss}(n) > 0\right\}$.

• Your computation is correct. Since the Markov chain is irreducible and $p_{22}>0$, it's clear that the period of both states is $1$ (that is, the chain is aperiodic). Aug 31 '16 at 9:55
• $p_{1,2}(1) = 1$, $p_{2,2}(1) =1/2$, $p_{2,2} = 1/2 + 1/2p_{1,2}=1/2+1/2 = 1$, $p_{1,1}(1) = 0$, $p_{1,1}(2) = 1\cdot p_{2,1}$ Lets try to find $o(1)=\gcd\{n:p_{1,1}(n) > 0\} = \gcd\{2,3,4,5,....\}=1$ $o(2) = \{1,2,3...\} = 1$. Tell me please, these compuations are ok ?
– user343207
Aug 31 '16 at 10:24
• That is correct, but it should be intuitively clear why the chain is aperiodic, from the definition, without need for explicit computation. Aug 31 '16 at 10:27
• Yes, I understand it. It is about your first comment. Thanks
– user343207
Aug 31 '16 at 10:37

Since the Markov chain is irreducible and aperiodic with a finite state space, we may also compute the stationary distribution $s$ by computing $\lim_{n\to\infty} P^n$ and taking any of its rows. Since $P^n$ is a stochastic matrix (i.e. the rows sum to $1$), it is clear that $(1,1)$ is an eigenvector with associated eigenvalue $1$. The sum of the eigenvalues of a matrix is equal to its trace (the sum of the diagonal elements), so the other eigenvalue is $-\frac12$. Solving $\left(P-\left(\frac12\right)I\right)x=0$ yields $(2,-1)$ as a solution. So we may write $P=ADA^{-1}$ where $$A = \begin{pmatrix}1&2\\1&-1\end{pmatrix},\quad D = \begin{pmatrix}1&0\\0&-\frac12\end{pmatrix}.$$ By induction we see that $P^n = AD^nA^{-1}$, and since $A^{-1}=\frac13 A$ we have $$P^n = \frac13\begin{pmatrix}1&2\\1&-1\end{pmatrix} \begin{pmatrix}1&0\\0&\left(-\frac12\right)^n\end{pmatrix}\begin{pmatrix}1&2\\1&-1\end{pmatrix}=\frac13\begin{pmatrix}1 - \left(\frac12\right)^{n-1}& 2\left(1-\left(\frac12\right)^n\right) \\ 1- \left(\frac12\right)^n & 2\left(1+\left(\frac12\right)^n\right)\end{pmatrix}.$$ It follows that $$\lim_{n\to\infty} P^n = \begin{pmatrix}\frac13&\frac23\\\frac13&\frac23, \end{pmatrix}$$ and so the stationary distribution is given by $$s = \left(\frac13,\frac23\right).$$
• Yes; consider the transition matrix $$P=\begin{pmatrix}0&1\\1&0\end{pmatrix}.$$ Then $\pi P=\pi$ implies $\pi_1=\pi_2$, so $\left(\frac12,\frac12\right)$ is the only stationary distribution. But clearly this Markov chain is periodic with period $2$. Aug 31 '16 at 19:37