Consider the transition matrix

$ P = \begin{bmatrix} 1-p&p\\ q&1-q \end{bmatrix} $

for general $2$-state Markov Chain $(0 \le p, q\le 1)$.

(a) Find the limiting distribution (if it exists) if $p + q = 1$.
(b) Find the limiting distribution (if it exists) if $p + q \ne 1$.

I myself calculated (a) with the answer $(\frac{q}{p+q}, \frac{p}{p+q}) = (q, p )$.

enter image description here

But, I couldn't understand the problem (b).

If $p+q \ne 1$, then the answer should be $(\frac{q}{p+q}, \frac{p}{p+q})$. Isn't that so?

What else could have been derived?

  • $\begingroup$ Well what if $p=q=1$ does the matrix converges $P^n$? $\endgroup$ – Phicar Apr 12 at 18:02
  • $\begingroup$ In your solution, did you actually use the fact that $p + q \neq 1$, other than in the very last step, when you resolve $(\frac{q}{p+q}, \frac{p}{p+q})$ to be $(p, q)$? I think you're right that the two solutions are nearly identical and can both be seen as specific cases of not placing any constraints on $p, q$ (besides $0 \leq p, q \leq 1)$. $\endgroup$ – Aaron Montgomery Apr 12 at 18:21
  • $\begingroup$ @AaronMontgomery, kindly, see the edit. $\endgroup$ – user366312 Apr 12 at 19:37
  • $\begingroup$ @Phicar, kindly, see the edit. $\endgroup$ – user366312 Apr 12 at 19:37


$$\begin{bmatrix} q&p \end{bmatrix} \begin{bmatrix} 1-p&p\\ q&1-q \end{bmatrix} =\begin{bmatrix} q - pq + pq & qp + p - pq \end{bmatrix} = \begin{bmatrix} q&p \end{bmatrix} $$

We have $v = [q\ \ p]$ is a right eigenvector with eigenvalue $1$, and the limiting distribution is a rescaled version of $v$ whose entries sum to $1$, i.e. $\pi = {v \over \sum_i v_i} = [{q \over p+q}\ \ {p \over p+q}]$.

None of the above depends on the value of the sum $p+q$; i.e. it is valid whether $p+q = 1$ or $= \sqrt{2}$ or $= 10^{-4}$ or any other value.

Nice trick question though!

Ooh, as @Phicar pointed out, the case of $p=q=1 \implies P^n$ does not converge. If you define the "limiting distribution" as $\lim P^n \pi_0$ then it does not exist. But if you define the "limiting distribution" as the unique probabilistic vector s.t. $\pi P = \pi$ then it does exist (and $\pi = [{q \over p+q}\ \ {p \over p+q}]$ as always). Not sure the terminology in this corner case...

Come to think of it, the $p=q=0$ case is even more problematic as any $\pi P = \pi$ (since $P=$ identity matrix).

  • $\begingroup$ have u seen the solution in the question? Can u kindly explain it? $\endgroup$ – user366312 Apr 12 at 19:42
  • $\begingroup$ the "solution" meaning the scanned handwritten note? i didn't check the details but that looks like a direct exact derivation of $P^n$ via eigen decomposition, maybe? anyway, if $0 < p+q < 2$ then $|1 - p - q| < 1$ and so $\lim (1 - p - q)^n = 0$ so all those terms vanish, and you can clearly see $P^\infty$ having two identical rows, both equal to $\pi$. $\endgroup$ – antkam Apr 12 at 19:49
  • $\begingroup$ @antkam Being limiting distribution implies being stationary. Not backwards. $\endgroup$ – Phicar Apr 12 at 19:51
  • $\begingroup$ @Phicar - thanks, I was not sure of the terminology (since I learned this literally decades ago). Then in the $p=q=1$ case there is no limiting distribution, while in the $p=q=0$ case every distribution is limiting. Right? $\endgroup$ – antkam Apr 12 at 19:55
  • $\begingroup$ Are we mixing up Limiting Distribution and Stationary Distribution? $\endgroup$ – user366312 Apr 12 at 20:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.