# What is the stationary distribution of the following Markov chain?

Consider a chain with state space $$\{1,2, \cdots \}.$$ If you are at $$1$$ go to state $$j$$ with probability $$p_j$$ $$( j=1,2,\cdots$$ $$) ,$$ where these are non-negative numbers adding to $$1$$. If you are in a state $$i>1\ ,$$ then go just one step back$$,$$ that is$$,$$ to $$i-1.$$ Discuss the nature of the states and nature of the stationary distribution$$.$$

Assuming that all $$p_j$$'s are positive I found that the Markov chain is irreducible with all it's states recurrent$$.$$ So there is no transient state and hence inessential state in the above Markov chain$$.$$ While calculating stationary distribution I found a problem$$.$$ Here it is $$:$$

Suppose $$\pi = (\pi_1 , \pi_2 , \cdots )$$ be the stationary distribution of the above Markov chain$$.$$ Then I found that $$\pi_2 = (1-p_1) \pi_1, \pi_3 = (1-p_1-p_2) \pi_1, \cdots$$ Since $$\sum\limits_{i=1}^{\infty} \pi_i = 1$$ so $$\pi_1 \{1 + (1-p_1) + (1-p_1 -p_2) + (1-p_1 -p_2 - p_3) + \cdots \}= 1.$$ Now how do I find the sum

$$1 + \sum_{k=1}^{\infty} (1-p_1-p_2 -p_3 - \cdots - p_k)\ ?$$

Or in other words

$$\sum\limits_{k=1}^{\infty} \sum\limits_{j=k}^{\infty} p_j\ ?$$

This gives us $$\pi_1$$ and consequently all the $$\pi_i$$'s$$.$$ where

$$\pi_i = \frac {\sum\limits_{k=i}^{\infty} p_k} {\sum\limits_{k=1}^{\infty} \sum\limits_{j=k}^{\infty} p_j}.$$

for $$i=1,2,\cdots.$$

• I get $\sum_{j=1}^\infty{jp_j}$ The value of this will depend on the $p_j$ of course. I think you've solved the problem already. Commented Dec 5, 2018 at 15:02
• Whose value is $\sum_{j=1}^{\infty} jp_j$? Commented Dec 5, 2018 at 15:12
• $1+(1-p_1)+(1-p_1-p_2)+(1-p_1-p_2-p_3)+\dots=\sum_{j=1}^{\infty} jp_j$ Commented Dec 5, 2018 at 15:17
• How? I got $$\sum\limits_{k=1}^{\infty} \sum\limits_{j=k}^{\infty} p_j.$$ You may check. How do you get your expression? Would you please share @saulspatz? Commented Dec 5, 2018 at 15:29
• Yeah I have understood. Anyway same expression. Actually I love multiple sums.ðŸ˜€ Commented Dec 5, 2018 at 15:30

\begin{align} \sum_{j=1}^\infty{jp_j}&=\sum_{j=1}^\infty{p_j}+\sum_{j=2}^\infty{(j-1)p_j}\\ &=1+\sum_{j=2}^\infty{p_j}+\sum_{j=3}^\infty{(j-2)p_j}\\ &=1+(1-p_1)+\sum_{j=3}^\infty{p_j}+\sum_{j=4}^\infty{(j-3)p_j}\\ &\vdots \end{align}
• @Dbchatto67 Right, I'd just finished typing it when I saw your comment, so I went ahead and posted it. An interesting point is that this problem shows if $p_j\ge0$ and $\sum{p_j}=1,$ then $\sum{jp_j}$ converges. Is there a more direct proof? Commented Dec 5, 2018 at 15:44