Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I have a markov chain with transition matrix below,

$$\begin{bmatrix} 1-q & q & & & \\ 1-q & 0 & q & & \\ & 1-q & 0 & q & \\ & & 1-q & 0 & q \\ & & & \ddots & \ddots & \ddots \end{bmatrix}$$ and I am asked to compute the stationary distribution for $q<\frac{1}{2}$. Using $\pi P =\pi$, I get that $\pi_n =\pi_0\left(\frac{q}{1-q}\right)^n$ and $\sum_{n=0}^\infty \pi_n = 1$.

Thus I get

$\pi_0\sum_{n=0}^\infty \left(\frac{q}{1-q}\right)^n=1 \\ \pi_0=\frac{1}{\sum_{n=0}^\infty \left(\frac{q}{1-q}\right)^n} \\ \text{thus } \pi_n = \frac{\left(\frac{q}{1-q}\right)^n}{\sum_{n=0}^\infty \left(\frac{q}{1-q}\right)^n}$

But it doesnt seem to be simplified. Is there anything else I can do to simplify $\pi_0=\frac{1}{\sum_{n=0}^\infty \left(\frac{q}{1-q}\right)^n}$? And thus simplify $\pi_n$.

share|improve this question
1  
Since $q < \frac{1}{2}$, $\gamma = \frac{q}{1-q} < 1$, so the sum in the denominator is a geometric series. –  Neal May 7 '12 at 19:49
    
Thanks @Neal, so I get that $\sum_{n=0}^\infty \left(\frac{q}{1-q}\right)^n =\frac{q}{1-2q}$, hence $\pi_0 = \frac{1-2q}{2q}$? –  Richard May 7 '12 at 20:07
    
I get $\sum\gamma^n = \frac{1}{1-\gamma} = \frac{1}{1-\frac{q}{1-q}} = \frac{1-q}{1-2q}$ so $\pi_0 = \frac{1-2q}{1-q}$. –  Neal May 10 '12 at 15:15
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.