A markov chain has an initial distribution $u^{(0)}$ = {1/6 1/2 1/3} and the following transition matrix

P= $\left(\begin{matrix}0&&0.5&&0.5\\0.5&&0&&0.5\\0.5&&0.5&&0\end{matrix}\right)$

Find its stationary distribution. Is it unique? Verify that the limiting distribution of the chain is stationary.

Since the matrix is doubly stochastic, the stationary distribution straight away works out to

($\pi_1$ $\pi_2$ $\pi_3$)= (1/3 1/3 1/3).... and also the distribution is unique.

But who do we verify that the limiting distribution is stationary?

  • $\begingroup$ Have you studied eigenvalues? Note, multiplying a vector all entries containing 1/3 with the matrix only yields the same vector, but does not prove that any initial state converges to that vector (although they ultimately do). With eigenvalues and eigenvectors you can prove both $\endgroup$ – imranfat Sep 22 '16 at 17:16
  • $\begingroup$ "But (how) do we verify that the limiting distribution is stationary?" By computing the distributions after $n$ steps and their limit. $\endgroup$ – Did Sep 22 '16 at 17:31
  • $\begingroup$ Thanks. I got the eigenvalues as 1,-0.5 and -0.5 and eigenvectors as (1 1 1) for 1 and (1, x, -x) for -0.5.How do I proceed from here? $\endgroup$ – SAK Sep 22 '16 at 17:32
  • $\begingroup$ OK.thanks..... to compute distribution after n steps and the limit do I calculate $p_{ij}^{(n)}$ for each of the nine values? $\endgroup$ – SAK Sep 22 '16 at 17:35
  • $\begingroup$ A shortcut is to note that $2P=J-I$ where $J$ is the matrix of ones. Then $J^2=3J$, $JI=IJ=J$ and $I^2=I$ hence $2^nP^n=a_nJ+b_nI$ where $(a_n,b_n)$ solves the recursion $a_{n+1}=2a_n+b_n$, $b_{n+1}=-b_n$, starting at $(a_0,b_0)=(0,1)$. Thus, $b_n=(-1)^{n}$ and $a_n=(2^n-(-1)^n)/3$ for every $n$, that is, $3P^n=J+(-1)^n(I-J)/2^n$, which proves that $P^n\to J/3$, as was expected. $\endgroup$ – Did Sep 28 '16 at 13:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.