1
$\begingroup$

Let $T = (X_n:n \in \mathbb{N})$ denote a homogeneous Markov chain with state space $E=\lbrace 1, 2, 3\rbrace$ and $$\mathbb{P}(X_1=2\vert X_0=1) = \mathbb{P}(X_1=3\vert X_0=1)=\frac{1}{3}$$ as well as $$\mathbb{P}(X_1=1\vert X_0=2) = \mathbb{P}(X_1=2\vert X_0=3)=1.$$

I suspect that the transition matrix is $$T = \left( \begin{array}{ccc} 1/3 & 1/3 & 1/3 \\ 1 & 0 & 0 \\ 0 & 1 & 0 \end{array} \right).$$

Also, I understand that the stationary distribution is a limiting form of the Markov chain, and so from the definition of a stationary distribution we find it $\pi$, the stationary distribution, by calculating $\pi=\pi T$.

We calculate the stationary distribution by finding left eigenvectors for the eigenvalue $1$. So, \begin{align*} \frac{1}{3}\pi_1 +\frac{1}{3}\pi_2+\frac{1}{3}\pi_3 & =\pi_1,\\ \pi_1 & =\pi_2 \hspace{1.0cm}\textrm{and}\\ \pi_2 & =\pi_3. \end{align*} So $\pi_1=\pi_2=\pi_3$ and as $\pi_1+\pi_2+\pi_3=1$, then $$\pi=\left(\frac{1}{3}, \frac{1}{3}, \frac{1}{3}\right).$$

Is this correct?

Also, positive recurrent Markov chain has a unique stationary distribution. How do I determine if this Markov chain is positive recurrent, and thus if the above stationary distribution is unique?

$\endgroup$
  • $\begingroup$ Transition matrices have row-sum 1 or column-sum 1, depending on the convention. Yours has neither. Re the stationary distribution, surely you know how to compute it starting from the transition matrix? $\endgroup$ – Did Oct 28 '13 at 16:40
  • $\begingroup$ You're right and I have amended the transition matrix. Hopefully it is now correct? $\endgroup$ – Levi Oct 29 '13 at 20:00
  • $\begingroup$ I'm still unsure of how to derive the stationary distribution. Any help on how to start it would be great. Thank you $\endgroup$ – Levi Oct 29 '13 at 20:01
  • 1
    $\begingroup$ For the answer you give, $\pi T\ne\pi$, so something must be wrong. The system of three equations you have written down give the solution of $T\pi=\pi$, which is a different kettle of fish altogether. $\endgroup$ – Gerry Myerson Nov 1 '13 at 22:37
  • 1
    $\begingroup$ As I wrote, you wanted the three equations for the system $\pi T=\pi$, but you wrote down the three equations for the system $T\pi=\pi$. $\endgroup$ – Gerry Myerson Nov 2 '13 at 3:54
1
$\begingroup$

Does this Markov chain have a unique stationary distribution? Or are there more than one? I know that a positive recurrent Markov chain has a unique distribution but I am not sure how to check if this Markov chain is positive recurrent.

The chain is irreducible on a finite state space hence it is positive recurrent and has a unique stationary distribution $\pi$. Following the precise indications in the comments, one gets that $\pi$ is in the proportions 3:2:1, thus $\pi=(\frac12,\frac13,\frac16)$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.