2
$\begingroup$

Let $T = (X_n:n \in \mathbb{N})$ denote a homogeneous Markov chain with state space $E=\lbrace 1, 2, 3\rbrace$ and $$\mathbb{P}(X_1=2\vert X_0=1) = \mathbb{P}(X_1=3\vert X_0=1)=\frac{1}{3}$$ as well as $$\mathbb{P}(X_1=1\vert X_0=2) = \mathbb{P}(X_1=2\vert X_0=3)=1.$$

I suspect that the transition matrix is $$T = \left( \begin{array}{ccc} 1/3 & 1/3 & 1/3 \\ 1 & 0 & 0 \\ 0 & 1 & 0 \end{array} \right).$$

Also, I understand that the stationary distribution is a limiting form of the Markov chain, and so from the definition of a stationary distribution we find it $\pi$, the stationary distribution, by calculating $\pi=\pi T$.

We calculate the stationary distribution by finding left eigenvectors for the eigenvalue $1$. So, \begin{align*} \frac{1}{3}\pi_1 +\frac{1}{3}\pi_2+\frac{1}{3}\pi_3 & =\pi_1,\\ \pi_1 & =\pi_2 \hspace{1.0cm}\textrm{and}\\ \pi_2 & =\pi_3. \end{align*} So $\pi_1=\pi_2=\pi_3$ and as $\pi_1+\pi_2+\pi_3=1$, then $$\pi=\left(\frac{1}{3}, \frac{1}{3}, \frac{1}{3}\right).$$

Is this correct?

Also, positive recurrent Markov chain has a unique stationary distribution. How do I determine if this Markov chain is positive recurrent, and thus if the above stationary distribution is unique?

$\endgroup$
13
  • $\begingroup$ Transition matrices have row-sum 1 or column-sum 1, depending on the convention. Yours has neither. Re the stationary distribution, surely you know how to compute it starting from the transition matrix? $\endgroup$
    – Did
    Commented Oct 28, 2013 at 16:40
  • $\begingroup$ You're right and I have amended the transition matrix. Hopefully it is now correct? $\endgroup$
    – Levi
    Commented Oct 29, 2013 at 20:00
  • $\begingroup$ I'm still unsure of how to derive the stationary distribution. Any help on how to start it would be great. Thank you $\endgroup$
    – Levi
    Commented Oct 29, 2013 at 20:01
  • 1
    $\begingroup$ For the answer you give, $\pi T\ne\pi$, so something must be wrong. The system of three equations you have written down give the solution of $T\pi=\pi$, which is a different kettle of fish altogether. $\endgroup$ Commented Nov 1, 2013 at 22:37
  • 1
    $\begingroup$ As I wrote, you wanted the three equations for the system $\pi T=\pi$, but you wrote down the three equations for the system $T\pi=\pi$. $\endgroup$ Commented Nov 2, 2013 at 3:54

1 Answer 1

2
$\begingroup$

Does this Markov chain have a unique stationary distribution? Or are there more than one? I know that a positive recurrent Markov chain has a unique distribution but I am not sure how to check if this Markov chain is positive recurrent.

The chain is irreducible on a finite state space hence it is positive recurrent and has a unique stationary distribution $\pi$. Following the precise indications in the comments, one gets that $\pi$ is in the proportions 3:2:1, thus $\pi=(\frac12,\frac13,\frac16)$.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .