1
$\begingroup$

Why is it that a non-negative integers Markov chain with transition prob. matrix (t.p.m) $P$ given by $p_{i,i+1} = p$ and $p_{i,0} = 1 − p$, has a unique stationary distribution $π$?

$\endgroup$
  • 1
    $\begingroup$ You are expected to show your effort. $\endgroup$ – Kavi Rama Murthy Nov 27 '18 at 5:57
2
$\begingroup$

Let $\pi$ denote a stationary distribution for the given Markov chain, assuming for the moment such a distribution exists. We may write $\pi$ as a vector, thus:

$\pi = (\pi_0, \pi_1, \pi_2, \ldots, \pi_k, \pi_{k + 1}, \ldots ); \tag 1$

also, $\pi$ must satisfy the normalization condition

$\displaystyle \sum_0^\infty \pi_k = 1, \tag 2$

which ensures that $\pi$ is in fact a probability distribution on the non-negative integers

$\Bbb Z_\ge = \{ z \in \Bbb Z, \; z \ge 0 \}. \tag 3$

Now it will be recalled that the equations

$p_{i, i + 1} = p, \; p_{i, 0} = 1 - p \tag 4$

define the conditional transition probabilities 'twixt the specified states ($i$ and $i + 1$, $i$ and $0$ according to the indices); thus the stationary probabilities $\pi_k$ must satisfy

$\pi_{i + 1} = p \pi_i, \tag 5$

and

$\pi_0 = (1 - p) \pi_i; \tag 6$

if we apply (5) repeatedly, starting with $i = 0$ we find

$\pi_1 = p \pi_0, \tag 7$

$\pi_2 = p\pi_1 = p^2 \pi_0, \tag 8$

and it is easy to see this pattern leads to

$\pi_k = p^k \pi_0, \; k \in \Bbb Z_\ge; \tag 9$

substituting this into the normalization equation (2) yields

$\pi_0 \displaystyle \sum_0^\infty p^k = \sum_0^\infty p^k \pi_0 = \sum_0^\infty \pi_k = 1, \tag{10}$

and since

$\displaystyle \sum_0^\infty p^k = \dfrac{1}{1 - p}, \; 0 < p < 1; \tag{11}$

we may solve (10) and obtain

$\pi_0 = 1 - p, \tag{12}$

and hence from (9),

$\pi_k = p^k(1 - p). \tag{13}$

As a consistency check, we may use (13) in the equation for $\pi_0$:

$\pi_0 = \displaystyle \sum_0^\infty (1 - p)\pi_i = \sum_0^\infty (1 - p)(1 - p)p^k = (1 - p)^2 \sum_0^\infty p^k = (1 - p)^2 \dfrac{1}{1 - p} = 1 - p. \tag{14}$

Now since a distribution $\pi$ as in (1) is given by (13), we see that a stationary distribution indeed exists; furthermore, it is clear that it is uniquely determined by (9)-(13). Thus the given Markov chain defined by (4) has a unique stationary distribution.

Nota Bene: Of course we should note that the above only applies subject to the condition

$0 < p < 1; \tag{15}$

if $p = 1$, then (6) and (9) show that

$\pi_k = 0, \; k \in \Bbb Z_\ge, \tag{16}$,

which, since such $\pi$ can't satisfy (2), is inadmissible as a probability distribution; if $p = 0$, (16) again follows from (5) and (6), and again we arrive at an inadmissible $\pi$. So we need to adopt (15) to obtain a unique and sensible result. End of Note.

$\endgroup$
  • 1
    $\begingroup$ You are God sent. Now I understand clearly. :) $\endgroup$ – Note Nov 27 '18 at 8:40
  • $\begingroup$ @Note: thanks for the kind words!!! $\endgroup$ – Robert Lewis Nov 27 '18 at 9:17
0
$\begingroup$

Algebraically you are trying to solve $\pi P = \pi$ (or equivalently $\pi(P-I) = 0$. Write out a couple of implied equations and you will see how to derive the solution and that it is unique...

$\endgroup$
  • 2
    $\begingroup$ The $i$th row of $P$ looks like $\begin{bmatrix} 1-p & 0 & 0 & \dots & p & 0 & \dots & 0 \end{bmatrix}$, where the $p$ is the $i+1$ position. The $i$th column then looks like $\begin{bmatrix} 0 \\ 0 \\ \dots \\ p \\ 0 \\ \dots \\ 0 \end{bmatrix}$ where now the $p$ is in the $i-1$ position, for $i>0$; for $i=0$ all the entries of the column are $1-p$. So the equations for the stationary distribution, for $i>0$, are $\pi_i=p \pi_{i-1}$. $\endgroup$ – Note Nov 27 '18 at 6:58
  • $\begingroup$ So how do I proceed from here $\endgroup$ – Note Nov 27 '18 at 7:07
  • $\begingroup$ Does $\pi = (p/(1-p), 1, 1)$ $\endgroup$ – Note Nov 27 '18 at 7:14

Not the answer you're looking for? Browse other questions tagged or ask your own question.