Consider the probability mass function $\pi(i)=\frac{7}{9}\big(\frac{2}{9}\big)^i$ for $i=0,1,2,3,...$ in $\{0,1,2,...\}$. How can I find a Markov matrix $P$ such that $\pi$ is its stationary distribution? Right now I have no clue other that trying to solve a bunch of linear equations and it seems too tedious. Maybe there is another way? Any help is appreciated!
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
The given pmf is that of a geometric distribution, i.e. $i$ consecutive successes before the first failure (or more usually, failures before success).
So one possible Markov chain is the one that describes a geometric distribution: advance from $i$ to $i+1$ consecutive successes with probability $1-q = p_{i,i+1}$, or return to $0$ with probability $q = p_{i,0}$.
To check this formally, solve the balance equations $\pi=\pi P$: Observe that for $i \in \{1,2,3,...\}$:
$$ \pi_i = \pi_{i-1} p_{i-1,i} = \pi_{i-1}(1-q) $$
Repeated substitution gives
$$ \pi_i = \pi_0(1-q)^i $$
and comparing this with your given expression for $\pi_i$, we get $q=\frac{7}{9}$. Note that for $i=0$,
$$ \pi_0 = \sum_{i=0}^\infty \pi_i p_{i,0} = \pi_0\sum_{i=0}^\infty (1-q)^i q = \pi_0 $$
i.e. it indeed holds (in fact, for any $0<q<1$).
