# What is the transition matrix and stationary distribution of ${T}$

Let $T = (X_n:n \in \mathbb{N})$ denote a homogeneous Markov chain with state space $E=\lbrace 1, 2, 3\rbrace$ and $$\mathbb{P}(X_1=2\vert X_0=1) = \mathbb{P}(X_1=3\vert X_0=1)=\frac{1}{3}$$ as well as $$\mathbb{P}(X_1=1\vert X_0=2) = \mathbb{P}(X_1=2\vert X_0=3)=1.$$

I suspect that the transition matrix is $$T = \left( \begin{array}{ccc} 1/3 & 1/3 & 1/3 \\ 1 & 0 & 0 \\ 0 & 1 & 0 \end{array} \right).$$

Also, I understand that the stationary distribution is a limiting form of the Markov chain, and so from the definition of a stationary distribution we find it $\pi$, the stationary distribution, by calculating $\pi=\pi T$.

We calculate the stationary distribution by finding left eigenvectors for the eigenvalue $1$. So, \begin{align*} \frac{1}{3}\pi_1 +\frac{1}{3}\pi_2+\frac{1}{3}\pi_3 & =\pi_1,\\ \pi_1 & =\pi_2 \hspace{1.0cm}\textrm{and}\\ \pi_2 & =\pi_3. \end{align*} So $\pi_1=\pi_2=\pi_3$ and as $\pi_1+\pi_2+\pi_3=1$, then $$\pi=\left(\frac{1}{3}, \frac{1}{3}, \frac{1}{3}\right).$$

Is this correct?

Also, positive recurrent Markov chain has a unique stationary distribution. How do I determine if this Markov chain is positive recurrent, and thus if the above stationary distribution is unique?

• Transition matrices have row-sum 1 or column-sum 1, depending on the convention. Yours has neither. Re the stationary distribution, surely you know how to compute it starting from the transition matrix? – Did Oct 28 '13 at 16:40
• You're right and I have amended the transition matrix. Hopefully it is now correct? – Levi Oct 29 '13 at 20:00
• I'm still unsure of how to derive the stationary distribution. Any help on how to start it would be great. Thank you – Levi Oct 29 '13 at 20:01
• For the answer you give, $\pi T\ne\pi$, so something must be wrong. The system of three equations you have written down give the solution of $T\pi=\pi$, which is a different kettle of fish altogether. – Gerry Myerson Nov 1 '13 at 22:37
• As I wrote, you wanted the three equations for the system $\pi T=\pi$, but you wrote down the three equations for the system $T\pi=\pi$. – Gerry Myerson Nov 2 '13 at 3:54

Does this Markov chain have a unique stationary distribution? Or are there more than one? I know that a positive recurrent Markov chain has a unique distribution but I am not sure how to check if this Markov chain is positive recurrent.

The chain is irreducible on a finite state space hence it is positive recurrent and has a unique stationary distribution $\pi$. Following the precise indications in the comments, one gets that $\pi$ is in the proportions 3:2:1, thus $\pi=(\frac12,\frac13,\frac16)$.