Consider the transition matrix

$ P = \begin{bmatrix} 1-p&p\\ q&1-q \end{bmatrix} $

for general $2$-state Markov Chain $(0 \le p, q\le 1)$.

  • Find the limiting distribution (if it exists) if $p + q \ne 1$.

Using mathematical induction, it is solved using the following proof:

$ p^n = \frac{1}{p+q} \begin{bmatrix}q&p\\q&p\end{bmatrix} + \frac{(1-p-q)^n}{p+q} \begin{bmatrix}p&-p\\-q&q\end{bmatrix} $

This is totally cumbersome.

Can this be solved in any other way like using $\pi (P-I) = 0$ and so on?

  • $\begingroup$ There are certainly many other ways to do this. However, if you understand what this decomposition represents, as I explained in an answer to your question about the decomposition, the inductive proof asked for in the exercise is actually quite simple and doesn’t involve tedious matrix computations. $\endgroup$ – amd Apr 14 at 21:47
  • $\begingroup$ @amd, I already talked about decomposition. I don't like it. Can u suggest any other easier alternative? $\endgroup$ – user366312 Apr 14 at 21:50

Assuming the eigenvalues are distinct, we can find two linearly independent eigenvectors. The result is that we can write $\mathbf{P}$ in the form $P = \mathbf{Q}\mathbf{D}\mathbf{Q}^{-1}$ where

$$ \mathbf{Q} = \left[\begin{array}{cc} 1&-p\\ 1&q \end{array} \right], \mathbf{Q}^{-1} = \left[\begin{array}{cc} \frac{q}{p+q}& \frac{p}{p+q}\\ \frac{-1}{p+q}&\frac{1}{p+q} \end{array} \right], \mathbf{D} = \left[\begin{array}{cc} 1&0\\ 0&1-p-q \end{array} \right] $$

Note that the columns of $Q$ are right-eigenvectors for $\mathbf{P}$, and the rows of $Q^{-1}$ are left-eigenvectors for $\mathbf{P}$.

The normalized left-eigenvector for eigenvalue 1 is:

$$\left[\begin{array}{c} \frac{q}{p+q} \\ \frac{p}{p+q} \end{array} \right]$$

so this is the stationary distribution. Now, since $0\leq p,q \leq 1$, if one is strictly in $(0, 1)$, the eigenvalue $1-p-q$ has $\left|1-p-q\right| \lt 1$. Therefore

$$ \mathbf{D} = \left[\begin{array}{cc} 1&0\\ 0&(1-p-q)^n \end{array} \right] \rightarrow \left[\begin{array}{cc} 1&0\\ 0&0 \end{array} \right], n \rightarrow \infty $$

And Also

$$ \mathbf{P} = \mathbf{Q}^{-1} \mathbf{D}^n \mathbf{Q} \rightarrow \left[\begin{array}{cc} q/(p+q)& p/(p+q)\\ -1/(p+q)& 1/(p+q) \end{array} \right] \left[\begin{array}{cc} 1&0\\ 0&0 \end{array} \right] \left[\begin{array}{cc} 1&-p\\ 1&p \end{array} \right] = \left[\begin{array}{cc} q/(p+q)& p/(p+q)\\ q/(p+q)& p/(p+q) \end{array} \right] = \left[\begin{array}{c} \pi\\ \pi \end{array} \right] $$

as $n \rightarrow \infty$. The chain converges to this stationary distribution regardless of initial distribution, provided at least one of $p, q$ is in $(0, 1)$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.