# Question on Markov-Chain GATE (ST)-$2021$

Question: Let $$\{X_n:n \ge0 \}$$ be a time- homogeneous discrete time Markov-chain with either finite or countable state space $$S$$. Then

$$1.$$ there is at least one recurrent state

$$2.$$ if there is an absorbing state, then there exists at least one stationary distribution

$$3.$$ if all states are positive recurrent, then there exists a unique stationary distribution

$$4.$$ If $$\{X_n:n \ge0 \}$$ is irreducible, $$S=\{1,2\}$$ and $$[\pi_1 , \pi_2]$$ is a stationary distribution, then $$\lim_{n\to \infty} P\{X_n=i|X_0=i\}=\pi_i,$$ for $$i=1,2.$$

My Attempt:

$$1.$$ I gave a counterexample. Take, $$P=\left [ \begin{matrix} 1/2 & 1/2 &0 &0& \cdots \\ 0 & 1/2 & 1/2&0 & \cdots \\ \vdots& \vdots & \vdots& \vdots& \ddots \\ \end{matrix} \right ]$$ with infinite state space $$S$$. Here, all states are transient.

$$2.$$ If there is an absorbing state, say state-$$i$$, so $$d(i)=1$$ and hence, state-$$i$$ is aperiodic. Since, every absorbing state is recurrent which implies Markov-chain has a recurrent, aperiodic state-$$i$$. I could not think further.

We first recall, "An irreducible positive recurrent Markov chain has a unique stationary distribution." See thisTheorem4,5.

$$3.$$ Take, $$P= \left [ \begin{matrix} 1&0 \\ 1/2&1/2\\ \end{matrix} \right ]$$. Here, all states are positive recurrent but it is infinitely stationary distributed.

$$4.$$ Really confused if we can apply theorem$$5$$.

The correct answer given in the key is $$2$$.

Kindly help me to prove this second option and help me understanding the last option( I am confused) and please check if my approach for other options is fine. Is there any use of the "homogeneous" thing here?

Thanks!

• I know that the chain given in option-$4$ is irreducible+finite $\implies$ positive recurrent and using above Theorem-$4$, we get a unique stationary distribution. Aug 4 at 5:59
• If $i$ is absorbing then $\pi_j=0$ for $j \neq i$ and $\pi_i=1$ defines a stationary distribution. Aug 4 at 6:03
• Hint for option $4$: Have a look at the chain with transition matrix $\ \pmatrix{0&1\\1&0}\$. Aug 4 at 20:42
• @lonza leggiera thanks, I got you. This chain is periodic with period $2$, therefore $\lim_{n\to \infty} P\{X_n=i|X_0=i\}=2 \pi_i \ne \pi_i$, Right? Aug 5 at 5:22
• Yes, you've got it. Aug 5 at 13:24

For $$(2)$$, using Kavi Rama Murty's hint given in the comments:
If there is an absorbing satate-$$i$$ $$\implies$$ $$i$$ is a periodic, recurrent. We get, $$\pi_j=0, \forall j\ne i$$, and $$\pi_i=1$$. This gives a stationary distribution.
For $$(4)$$, use lonza leggiera's hint:
Given $$\{X_n: n\ge0\}$$ is irreducible with finite state space $$S=\{1,2\}$$ and it has a stationary distribution $$\pi= [\pi_1, \pi_2]$$. Note that the given chain may not be aperiodic, so we take a periodic chain with transition probability matrix $$P=\left[ \begin{matrix} 0&1 \\ 1 &0\end{matrix} \right ]$$. Here, $$\left[ \begin{matrix} 1/2&1/2 \end{matrix} \right ]$$ is a stationary distribution but this chain does not have a limiting distribution.