# Markov chain period

Let a Markov chain with State space $E=\{1,2,3,4\}$ and probability transition matrix:

$$P=\begin{bmatrix} 0 & 1 & 0 & 0 \\ 1/4 & 0 & 1/4 & 1/2\\0 & 1& 0 & 0 \\ 0 &1/2&0&1/2 \end{bmatrix}$$

How can I find the period of the chain?

I think the states 1,2,3 are periodic with period 2 and 4 is aperiodic because there is a loop.

• What have you tried? Have you drawn a diagram of the chain? Which states do you think are periodic and which of them are aperiodic?
– saz
May 24, 2015 at 14:53
• This is not a transition matrix.
– Did
May 24, 2015 at 14:59
• @GiulyB The chain is aperiodic if all states are aperiodic and so far we do not know this, right? Okay, so there is a loop at state $4$ and therefore $4$ is aperiodic, I agree. What about the remaining ones? Any idea how to prove that they have period 2?
– saz
May 24, 2015 at 15:31
• @GiulyB Well, what does this tell us? A state $i$ is (in general) not periodic with period $2$ if $p^2(i,i)>0$. What do you really have to show?
– saz
May 24, 2015 at 15:57
• What you have shown (in your last comment) is that $2 \in N_x$ for all $x \in \{1,2,3,4\}$. This implies $d_x = gcd(N_x) \in \{1,2\}$. It remains to decide whether $d_x = 1$ or $d_x = 2$ for $x \in \{1,2,3\}$.
– saz
May 24, 2015 at 16:29

Set $\DeclareMathOperator \gcd{gcd}$

$$N_x := \{n \in \mathbb{N}; p^n(x,x)>0\} \qquad \quad d_x := \gcd(N_x).$$

• state 4: Since $p(4,4) = \frac{1}{2}>0$, it follows that $p^n(4,4)>0$ for all $n \in \mathbb{N}$; hence, $N_4 = \mathbb{N}$ and $d_4 = \gcd(\mathbb{N})=1$, i.e. state $4$ is aperiodic.
• state 3: We have $$p^2(3,3) = \mathbb{P}^3(X_2 = 3) \geq \mathbb{P}^3(X_2 = 3, X_1 = 2) = 1 \cdot \frac{1}{4} >0,$$ i.e. $2 \in N_3$. Moreover, \begin{align*} p^5(3,3) &= \mathbb{P}^3(X_5 = 3) \\ &\geq \mathbb{P}^3(X_1 = 2, X_2 = 4, X_3 = 4, X_4 = 2, X_5 = 3) \\ &= 1 \cdot \frac{1}{2} \cdot \frac{1}{2} \cdot \frac{1}{2} \cdot \frac{1}{4} >0. \end{align*} Consequently, $5 \in N_3$ and this implies $d_3 \leq \gcd(2,5)= 1$.

The argumentation for state 1 and 2 is similar, I leave it to you.

• so which is the period of the chain? May 24, 2015 at 16:42
• @GiulyB What conclusion did you draw for state 1 and 2? I won't solve your homework for you.
– saz
May 24, 2015 at 16:45
• also $d_2\leq gdc(2,5)$ and $d_1\leq gdc(2,5)$.All the states have period 1 so the chain is aperiodic. May 24, 2015 at 17:02
• @GiulyB Yes, that's correct.
– saz
May 24, 2015 at 17:27