0
$\begingroup$

Question: Let $\{X_n:n \ge0 \}$ be a time- homogeneous discrete time Markov-chain with either finite or countable state space $S$. Then

$1.$ there is at least one recurrent state

$2.$ if there is an absorbing state, then there exists at least one stationary distribution

$3.$ if all states are positive recurrent, then there exists a unique stationary distribution

$4.$ If $\{X_n:n \ge0 \}$ is irreducible, $S=\{1,2\}$ and $[\pi_1 , \pi_2]$ is a stationary distribution, then $\lim_{n\to \infty} P\{X_n=i|X_0=i\}=\pi_i,$ for $i=1,2.$

My Attempt:

$1.$ I gave a counterexample. Take, $P=\left [ \begin{matrix} 1/2 & 1/2 &0 &0& \cdots \\ 0 & 1/2 & 1/2&0 & \cdots \\ \vdots& \vdots & \vdots& \vdots& \ddots \\ \end{matrix} \right ]$ with infinite state space $S$. Here, all states are transient.

$2.$ If there is an absorbing state, say state-$i$, so $d(i)=1$ and hence, state-$i$ is aperiodic. Since, every absorbing state is recurrent which implies Markov-chain has a recurrent, aperiodic state-$i$. I could not think further.

We first recall, "An irreducible positive recurrent Markov chain has a unique stationary distribution." See thisTheorem4,5.

$3.$ Take, $P= \left [ \begin{matrix} 1&0 \\ 1/2&1/2\\ \end{matrix} \right ]$. Here, all states are positive recurrent but it is infinitely stationary distributed.

$4.$ Really confused if we can apply theorem$5$.

The correct answer given in the key is $2$.

Kindly help me to prove this second option and help me understanding the last option( I am confused) and please check if my approach for other options is fine. Is there any use of the "homogeneous" thing here?

Thanks!

$\endgroup$
10
  • $\begingroup$ I know that the chain given in option-$4$ is irreducible+finite $\implies $ positive recurrent and using above Theorem-$4$, we get a unique stationary distribution. $\endgroup$
    – Learning
    Aug 4, 2021 at 5:59
  • 1
    $\begingroup$ If $i$ is absorbing then $\pi_j=0$ for $j \neq i$ and $\pi_i=1$ defines a stationary distribution. $\endgroup$ Aug 4, 2021 at 6:03
  • 1
    $\begingroup$ Hint for option $4$: Have a look at the chain with transition matrix $\ \pmatrix{0&1\\1&0}\ $. $\endgroup$ Aug 4, 2021 at 20:42
  • $\begingroup$ @lonza leggiera thanks, I got you. This chain is periodic with period $2$, therefore $\lim_{n\to \infty} P\{X_n=i|X_0=i\}=2 \pi_i \ne \pi_i$, Right? $\endgroup$
    – Learning
    Aug 5, 2021 at 5:22
  • 1
    $\begingroup$ Yes, you've got it. $\endgroup$ Aug 5, 2021 at 13:24

1 Answer 1

1
$\begingroup$

I have got the answers but I am answering to remove this question from unanswered que and for others if someone needs the answer in future.

For $(2)$, using Kavi Rama Murty's hint given in the comments:

If there is an absorbing satate-$i$ $\implies$ $i$ is a periodic, recurrent. We get, $\pi_j=0, \forall j\ne i$, and $\pi_i=1$. This gives a stationary distribution.

For $(4)$, use lonza leggiera's hint:

Given $\{X_n: n\ge0\}$ is irreducible with finite state space $S=\{1,2\}$ and it has a stationary distribution $\pi= [\pi_1, \pi_2]$. Note that the given chain may not be aperiodic, so we take a periodic chain with transition probability matrix $P=\left[ \begin{matrix} 0&1 \\ 1 &0\end{matrix} \right ]$. Here, $\left[ \begin{matrix} 1/2&1/2 \end{matrix} \right ]$ is a stationary distribution but this chain does not have a limiting distribution.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .