Suppose that ${X_n}$ ; $n≥0$ is an irreducible Markov chain on $S$ with stationary distribution $π = (π(i))$ $i∈S $ and let $Y_n$ = $(X_n, X_{n+1})$ be the Markov chain on $S'$ = ${(i, j) : P_{i,j} > 0}$ $⊂ S×S$.

a) Let $P'$ be the transition matrix for the Markov chain ${Y_n}$ ; $n≥0$. Compute the matrix $P'$ in the special case when ${X_n}$;$n≥0$ is the Markov chain on $S$ = $[0, 1]$ with transition matrix P = $$\begin{pmatrix} 1/4 & 3/4 \\ 1/2& 1/2 \\ \end{pmatrix}$$

b) Compute the stationary distribution $π'$ for the Markov chain ${Y_n}$; $n≥0$ in the special case of the part above.

c) Generalize your answer from part b) by showing that it is always true that $π'$ = ($π'(i, j))_{(i,j)∈S'}$ given by $π'(i, j)$ = $π(i)P_{i,j}$ is a stationary distribution for ${Y_n}$;$n≥0$.

What I have done so far: a) The transition matrix that I got for $Y_{n}$ is $$\begin{pmatrix} 1/4 & 3/4 &0 &0\\ 0&0 &1/2 &1/2\\ 1/4&3/4 &0 &0\\ 0&0 &1/2&1/2 \\ \end{pmatrix}$$ where the the rows and columns are in order of $00,01,10,11$

b) From the transition matrix - I have four equations for the stationary distribution which in the end gives me $$π_{1} =π_{2}=π_{3}=π_{4} $$ and therefore my $π$ = ${1/10,3/10,3/10,3/10}$ ( I am not sure whether it is correct or not - it feels weird to me)

c) I am having difficulty understanding what it is saying.


1 Answer 1


To understand what is required to prove in part (c) it is firstly impotrant to use the same notations as in the body of the question. Let $\pi'=(\pi'(0,0),\,\pi'(0,1),\, \pi'(1,0),\,\pi'(1,1))$ is a stationary distribution for $Y_n$.

(b) You solved $\pi'=P'\pi'$, this is not the system of equations for determine stationary distribution. For any stochastic matrices this system of equations gives the vector with the same coordinates.

If $\pi'$ is an initial distribution, the distribution on the next step is $\pi'\cdot P'$, so the equality $\pi'=\pi' P'$ defines stationary distribution. It is $$ (\pi'(0,0),\,\pi'(0,1),\, \pi'(1,0),\,\pi'(1,1)) = \left(0.1,\,0.3,\,0.3,\,0.3\right). $$

Return to (c). You need firstly to find stationary distribution $\pi=(\pi(0),\pi(1))$ for initial chain $X_n$ by $\pi=\pi P$, and check whether $\pi'(i,j)=\pi(i) P_{i,j}$ that is $$ 0.1=\pi'(0,0) = \pi(0)\cdot P_{0,0}=\pi(0)\cdot\frac14, $$ $$ 0.3=\pi'(0,1) = \pi(0)\cdot P_{0,1}=\pi(0)\cdot \frac34, $$ $$ 0.3=\pi'(1,0) = \pi(1)\cdot P_{1,0}=\pi(1)\cdot \frac12 $$ and $$ 0.3=\pi'(1,1) = \pi(1)\cdot P_{1,1}=\pi(1)\cdot \frac12. $$

And then there are two possibilities for understand what generalizations are needed in part (c).

Either you generalize it for arbitrary transition matrix $P=\pmatrix{a & 1-a\\ 1-b & b}$ on state space $S=\{0,1\}$ and repeat all the steps from the beginning: write $P'$, find stationary distribution $\pi'$ for it, find stationary distribution $\pi$ and check whether $\pi'(i,j)=\pi(i) P_{i,j}$ for all $i,j\in\{0,1\}$,

or (which looks more probable for me), suppose that $S$ is arbitrary finite state space, $P$ is a transition matrix for $X_n$ for which there exists stationary distribution $\pi$, $Y_n=(X_n,X_{n+1})$ is a MC on $S\times S$, then stationary distribution $\pi'$ satisfies $\pi'(i,j)=\pi(i) P_{i,j}$ for all $i,j\in S$.

For the last case, there is no need to write $P'$. Stationary distribution satisfies the property: if the chain starts from stationary distribution, it stays in stationary distribution for any step. So the only thing you need to check is :

Let $\mathbb P(X_0=i,X_1=j)=\pi'(i,j)=\mathbb P(X_1=i,X_2=j)$ for all $i,j$ then $\pi'(i,j)=\pi(i) P_{i,j}$.

And this can easily be checked: $$ \pi'(i,j)=\mathbb P(X_0=i,X_1=j)= \mathbb P(X_0=i)\mathbb P(X_1=j\mid X_0=i) = \mathbb P(X_0=i) P_{i,j} $$ $$ \pi'(i,j)=\mathbb P(X_1=i,X_2=j)= \mathbb P(X_1=i)\mathbb P(X_2=j\mid X_1=i)=P(X_1=i) P_{i,j} $$ and these probabilities coincide iff $P(X_0=i)=P(X_1=i)$ for all $i$, so $X_n$ works in stationary distribution $P(X_n=i)=\pi(i)$. And then $$ \pi'(i,j) = \mathbb P(X_0=i) P_{i,j} = \pi(i)P_{i,j}. $$


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .