1
$\begingroup$

I have a problem for which the Markov matrix turns out to be the following:

$$P = \begin{pmatrix} 0 & 0.5 & 0 & 0.5\\ 0.5 & 0 & 0.5 & 0 \\ 0 & 0.5 & 0 & 0.5\\ 0.5 & 0 & 0.5 & 0\\ \end{pmatrix}$$

This matrix has eigenvalues $-1$ and $1$. Hence, it does not converge to some matrix $A$ when raised to some power $n$. However, there exists a solution to $Pv = v$. This is basically, the eigenvector corresponding to the eigenvalue of $1$. Is $v$ the steady state probabilites?

$\endgroup$

1 Answer 1

2
$\begingroup$

This Markov chain is periodic with period $2$, meaning that for any initial state $i$, the limiting probability $\lim_{n\to\infty}\mathbb P(X_n=j\mid X_0=i)$ does not exist. However, $\{X_n\}$ is irreducible, positive recurrent, and has finitely many states, so a unique stationary distribution $\pi$ exists. Since $P$ is doubly stochastic (both the rows and the columns sum to one), it follows that $\pi$ is the uniform distribution over $0,1,2,3$, i.e. $\pi_0=\pi_1=\pi_2=\pi_3=\frac14$.

Now, because $\{X_n\}$ is periodic with period $2$, the limits of $P^{2n}$ and $P^{2n+1}$ as $n\to\infty$ exist. In particular, $$ P^{2n} = \left( \begin{array}{cccc} \frac{1}{2} & 0 & \frac{1}{2} & 0 \\ 0 & \frac{1}{2} & 0 & \frac{1}{2} \\ \frac{1}{2} & 0 & \frac{1}{2} & 0 \\ 0 & \frac{1}{2} & 0 & \frac{1}{2} \\ \end{array} \right),\quad P^{2n+1} = P $$ for all $n$, so $$ \lim_{n\to\infty} P^{2n} = \left( \begin{array}{cccc} \frac{1}{2} & 0 & \frac{1}{2} & 0 \\ 0 & \frac{1}{2} & 0 & \frac{1}{2} \\ \frac{1}{2} & 0 & \frac{1}{2} & 0 \\ 0 & \frac{1}{2} & 0 & \frac{1}{2} \\ \end{array} \right),\quad \lim_{n\to\infty}P^{2n+1} = P. $$

$\endgroup$
0

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .