Question: Let $\{X_n:n \ge0 \}$ be a time- homogeneous discrete time Markov-chain with either finite or countable state space $S$. Then

$1.$ there is at least one recurrent state

$2.$ if there is an absorbing state, then there exists at least one stationary distribution

$3.$ if all states are positive recurrent, then there exists a unique stationary distribution

$4.$ If $\{X_n:n \ge0 \}$ is irreducible, $S=\{1,2\}$ and $[\pi_1 , \pi_2]$ is a stationary distribution, then $\lim_{n\to \infty} P\{X_n=i|X_0=i\}=\pi_i,$ for $i=1,2.$

My Attempt:

$1.$ I gave a counterexample. Take, $P=\left [ \begin{matrix} 1/2 & 1/2 &0 &0& \cdots \\ 0 & 1/2 & 1/2&0 & \cdots \\ \vdots& \vdots & \vdots& \vdots& \ddots \\ \end{matrix} \right ]$ with infinite state space $S$. Here, all states are transient.

$2.$ If there is an absorbing state, say state-$i$, so $d(i)=1$ and hence, state-$i$ is aperiodic. Since, every absorbing state is recurrent which implies Markov-chain has a recurrent, aperiodic state-$i$. I could not think further.

We first recall, "An irreducible positive recurrent Markov chain has a unique stationary distribution." See thisTheorem4,5.

$3.$ Take, $P= \left [ \begin{matrix} 1&0 \\ 1/2&1/2\\ \end{matrix} \right ]$. Here, all states are positive recurrent but it is infinitely stationary distributed.

$4.$ Really confused if we can apply theorem$5$.

The correct answer given in the key is $2$.

Kindly help me to prove this second option and help me understanding the last option( I am confused) and please check if my approach for other options is fine. Is there any use of the "homogeneous" thing here?


  • $\begingroup$ I know that the chain given in option-$4$ is irreducible+finite $\implies $ positive recurrent and using above Theorem-$4$, we get a unique stationary distribution. $\endgroup$
    – Learning
    Aug 4 at 5:59
  • 1
    $\begingroup$ If $i$ is absorbing then $\pi_j=0$ for $j \neq i$ and $\pi_i=1$ defines a stationary distribution. $\endgroup$ Aug 4 at 6:03
  • 1
    $\begingroup$ Hint for option $4$: Have a look at the chain with transition matrix $\ \pmatrix{0&1\\1&0}\ $. $\endgroup$ Aug 4 at 20:42
  • $\begingroup$ @lonza leggiera thanks, I got you. This chain is periodic with period $2$, therefore $\lim_{n\to \infty} P\{X_n=i|X_0=i\}=2 \pi_i \ne \pi_i$, Right? $\endgroup$
    – Learning
    Aug 5 at 5:22
  • 1
    $\begingroup$ Yes, you've got it. $\endgroup$ Aug 5 at 13:24

I have got the answers but I am answering to remove this question from unanswered que and for others if someone needs the answer in future.

For $(2)$, using Kavi Rama Murty's hint given in the comments:

If there is an absorbing satate-$i$ $\implies$ $i$ is a periodic, recurrent. We get, $\pi_j=0, \forall j\ne i$, and $\pi_i=1$. This gives a stationary distribution.

For $(4)$, use lonza leggiera's hint:

Given $\{X_n: n\ge0\}$ is irreducible with finite state space $S=\{1,2\}$ and it has a stationary distribution $\pi= [\pi_1, \pi_2]$. Note that the given chain may not be aperiodic, so we take a periodic chain with transition probability matrix $P=\left[ \begin{matrix} 0&1 \\ 1 &0\end{matrix} \right ]$. Here, $\left[ \begin{matrix} 1/2&1/2 \end{matrix} \right ]$ is a stationary distribution but this chain does not have a limiting distribution.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.