# Finding limiting distribution.

Consider the transition matrix

$$P = \begin{bmatrix} 1-p&p\\ q&1-q \end{bmatrix}$$

for general $$2$$-state Markov Chain $$(0 \le p, q\le 1)$$.

(a) Find the limiting distribution (if it exists) if $$p + q = 1$$.
(b) Find the limiting distribution (if it exists) if $$p + q \ne 1$$.

I myself calculated (a) with the answer $$(\frac{q}{p+q}, \frac{p}{p+q}) = (q, p )$$.

But, I couldn't understand the problem (b).

If $$p+q \ne 1$$, then the answer should be $$(\frac{q}{p+q}, \frac{p}{p+q})$$. Isn't that so?

What else could have been derived?

• Well what if $p=q=1$ does the matrix converges $P^n$? – Phicar Apr 12 at 18:02
• In your solution, did you actually use the fact that $p + q \neq 1$, other than in the very last step, when you resolve $(\frac{q}{p+q}, \frac{p}{p+q})$ to be $(p, q)$? I think you're right that the two solutions are nearly identical and can both be seen as specific cases of not placing any constraints on $p, q$ (besides $0 \leq p, q \leq 1)$. – Aaron Montgomery Apr 12 at 18:21
• @AaronMontgomery, kindly, see the edit. – user366312 Apr 12 at 19:37
• @Phicar, kindly, see the edit. – user366312 Apr 12 at 19:37

Since:

$$\begin{bmatrix} q&p \end{bmatrix} \begin{bmatrix} 1-p&p\\ q&1-q \end{bmatrix} =\begin{bmatrix} q - pq + pq & qp + p - pq \end{bmatrix} = \begin{bmatrix} q&p \end{bmatrix}$$

We have $$v = [q\ \ p]$$ is a right eigenvector with eigenvalue $$1$$, and the limiting distribution is a rescaled version of $$v$$ whose entries sum to $$1$$, i.e. $$\pi = {v \over \sum_i v_i} = [{q \over p+q}\ \ {p \over p+q}]$$.

None of the above depends on the value of the sum $$p+q$$; i.e. it is valid whether $$p+q = 1$$ or $$= \sqrt{2}$$ or $$= 10^{-4}$$ or any other value.

Nice trick question though!

Ooh, as @Phicar pointed out, the case of $$p=q=1 \implies P^n$$ does not converge. If you define the "limiting distribution" as $$\lim P^n \pi_0$$ then it does not exist. But if you define the "limiting distribution" as the unique probabilistic vector s.t. $$\pi P = \pi$$ then it does exist (and $$\pi = [{q \over p+q}\ \ {p \over p+q}]$$ as always). Not sure the terminology in this corner case...

Come to think of it, the $$p=q=0$$ case is even more problematic as any $$\pi P = \pi$$ (since $$P=$$ identity matrix).

• have u seen the solution in the question? Can u kindly explain it? – user366312 Apr 12 at 19:42
• the "solution" meaning the scanned handwritten note? i didn't check the details but that looks like a direct exact derivation of $P^n$ via eigen decomposition, maybe? anyway, if $0 < p+q < 2$ then $|1 - p - q| < 1$ and so $\lim (1 - p - q)^n = 0$ so all those terms vanish, and you can clearly see $P^\infty$ having two identical rows, both equal to $\pi$. – antkam Apr 12 at 19:49
• @antkam Being limiting distribution implies being stationary. Not backwards. – Phicar Apr 12 at 19:51
• @Phicar - thanks, I was not sure of the terminology (since I learned this literally decades ago). Then in the $p=q=1$ case there is no limiting distribution, while in the $p=q=0$ case every distribution is limiting. Right? – antkam Apr 12 at 19:55
• Are we mixing up Limiting Distribution and Stationary Distribution? – user366312 Apr 12 at 20:03