# Markov Chain with Unique Stationary Distribution [closed]

Why is it that a non-negative integers Markov chain with transition prob. matrix (t.p.m) $$P$$ given by $$p_{i,i+1} = p$$ and $$p_{i,0} = 1 − p$$, has a unique stationary distribution $$π$$?

• You are expected to show your effort. – Kavi Rama Murthy Nov 27 '18 at 5:57

Let $$\pi$$ denote a stationary distribution for the given Markov chain, assuming for the moment such a distribution exists. We may write $$\pi$$ as a vector, thus:

$$\pi = (\pi_0, \pi_1, \pi_2, \ldots, \pi_k, \pi_{k + 1}, \ldots ); \tag 1$$

also, $$\pi$$ must satisfy the normalization condition

$$\displaystyle \sum_0^\infty \pi_k = 1, \tag 2$$

which ensures that $$\pi$$ is in fact a probability distribution on the non-negative integers

$$\Bbb Z_\ge = \{ z \in \Bbb Z, \; z \ge 0 \}. \tag 3$$

Now it will be recalled that the equations

$$p_{i, i + 1} = p, \; p_{i, 0} = 1 - p \tag 4$$

define the conditional transition probabilities 'twixt the specified states ($$i$$ and $$i + 1$$, $$i$$ and $$0$$ according to the indices); thus the stationary probabilities $$\pi_k$$ must satisfy

$$\pi_{i + 1} = p \pi_i, \tag 5$$

and

$$\pi_0 = (1 - p) \pi_i; \tag 6$$

if we apply (5) repeatedly, starting with $$i = 0$$ we find

$$\pi_1 = p \pi_0, \tag 7$$

$$\pi_2 = p\pi_1 = p^2 \pi_0, \tag 8$$

and it is easy to see this pattern leads to

$$\pi_k = p^k \pi_0, \; k \in \Bbb Z_\ge; \tag 9$$

substituting this into the normalization equation (2) yields

$$\pi_0 \displaystyle \sum_0^\infty p^k = \sum_0^\infty p^k \pi_0 = \sum_0^\infty \pi_k = 1, \tag{10}$$

and since

$$\displaystyle \sum_0^\infty p^k = \dfrac{1}{1 - p}, \; 0 < p < 1; \tag{11}$$

we may solve (10) and obtain

$$\pi_0 = 1 - p, \tag{12}$$

and hence from (9),

$$\pi_k = p^k(1 - p). \tag{13}$$

As a consistency check, we may use (13) in the equation for $$\pi_0$$:

$$\pi_0 = \displaystyle \sum_0^\infty (1 - p)\pi_i = \sum_0^\infty (1 - p)(1 - p)p^k = (1 - p)^2 \sum_0^\infty p^k = (1 - p)^2 \dfrac{1}{1 - p} = 1 - p. \tag{14}$$

Now since a distribution $$\pi$$ as in (1) is given by (13), we see that a stationary distribution indeed exists; furthermore, it is clear that it is uniquely determined by (9)-(13). Thus the given Markov chain defined by (4) has a unique stationary distribution.

Nota Bene: Of course we should note that the above only applies subject to the condition

$$0 < p < 1; \tag{15}$$

if $$p = 1$$, then (6) and (9) show that

$$\pi_k = 0, \; k \in \Bbb Z_\ge, \tag{16}$$,

which, since such $$\pi$$ can't satisfy (2), is inadmissible as a probability distribution; if $$p = 0$$, (16) again follows from (5) and (6), and again we arrive at an inadmissible $$\pi$$. So we need to adopt (15) to obtain a unique and sensible result. End of Note.

• You are God sent. Now I understand clearly. :) – Note Nov 27 '18 at 8:40
• @Note: thanks for the kind words!!! – Robert Lewis Nov 27 '18 at 9:17

Algebraically you are trying to solve $$\pi P = \pi$$ (or equivalently $$\pi(P-I) = 0$$. Write out a couple of implied equations and you will see how to derive the solution and that it is unique...

• The $i$th row of $P$ looks like $\begin{bmatrix} 1-p & 0 & 0 & \dots & p & 0 & \dots & 0 \end{bmatrix}$, where the $p$ is the $i+1$ position. The $i$th column then looks like $\begin{bmatrix} 0 \\ 0 \\ \dots \\ p \\ 0 \\ \dots \\ 0 \end{bmatrix}$ where now the $p$ is in the $i-1$ position, for $i>0$; for $i=0$ all the entries of the column are $1-p$. So the equations for the stationary distribution, for $i>0$, are $\pi_i=p \pi_{i-1}$. – Note Nov 27 '18 at 6:58
• So how do I proceed from here – Note Nov 27 '18 at 7:07
• Does $\pi = (p/(1-p), 1, 1)$ – Note Nov 27 '18 at 7:14