# How can I compute the Limiting Distribution in the following problem?

Consider the transition matrix

$$P = \begin{bmatrix} 1-p&p\\ q&1-q \end{bmatrix}$$

for general $$2$$-state Markov Chain $$(0 \le p, q\le 1)$$.

• Find the limiting distribution (if it exists) if $$p + q \ne 1$$.

Using mathematical induction, it is solved using the following proof:

$$p^n = \frac{1}{p+q} \begin{bmatrix}q&p\\q&p\end{bmatrix} + \frac{(1-p-q)^n}{p+q} \begin{bmatrix}p&-p\\-q&q\end{bmatrix}$$

This is totally cumbersome.

Can this be solved in any other way like using $$\pi (P-I) = 0$$ and so on?

• There are certainly many other ways to do this. However, if you understand what this decomposition represents, as I explained in an answer to your question about the decomposition, the inductive proof asked for in the exercise is actually quite simple and doesn’t involve tedious matrix computations. – amd Apr 14 at 21:47
• @amd, I already talked about decomposition. I don't like it. Can u suggest any other easier alternative? – user366312 Apr 14 at 21:50

Assuming the eigenvalues are distinct, we can find two linearly independent eigenvectors. The result is that we can write $$\mathbf{P}$$ in the form $$P = \mathbf{Q}\mathbf{D}\mathbf{Q}^{-1}$$ where

$$\mathbf{Q} = \left[\begin{array}{cc} 1&-p\\ 1&q \end{array} \right], \mathbf{Q}^{-1} = \left[\begin{array}{cc} \frac{q}{p+q}& \frac{p}{p+q}\\ \frac{-1}{p+q}&\frac{1}{p+q} \end{array} \right], \mathbf{D} = \left[\begin{array}{cc} 1&0\\ 0&1-p-q \end{array} \right]$$

Note that the columns of $$Q$$ are right-eigenvectors for $$\mathbf{P}$$, and the rows of $$Q^{-1}$$ are left-eigenvectors for $$\mathbf{P}$$.

The normalized left-eigenvector for eigenvalue 1 is:

$$\left[\begin{array}{c} \frac{q}{p+q} \\ \frac{p}{p+q} \end{array} \right]$$

so this is the stationary distribution. Now, since $$0\leq p,q \leq 1$$, if one is strictly in $$(0, 1)$$, the eigenvalue $$1-p-q$$ has $$\left|1-p-q\right| \lt 1$$. Therefore

$$\mathbf{D} = \left[\begin{array}{cc} 1&0\\ 0&(1-p-q)^n \end{array} \right] \rightarrow \left[\begin{array}{cc} 1&0\\ 0&0 \end{array} \right], n \rightarrow \infty$$

And Also

$$\mathbf{P} = \mathbf{Q}^{-1} \mathbf{D}^n \mathbf{Q} \rightarrow \left[\begin{array}{cc} q/(p+q)& p/(p+q)\\ -1/(p+q)& 1/(p+q) \end{array} \right] \left[\begin{array}{cc} 1&0\\ 0&0 \end{array} \right] \left[\begin{array}{cc} 1&-p\\ 1&p \end{array} \right] = \left[\begin{array}{cc} q/(p+q)& p/(p+q)\\ q/(p+q)& p/(p+q) \end{array} \right] = \left[\begin{array}{c} \pi\\ \pi \end{array} \right]$$

as $$n \rightarrow \infty$$. The chain converges to this stationary distribution regardless of initial distribution, provided at least one of $$p, q$$ is in $$(0, 1)$$.