0
$\begingroup$

Consider a chain with state space $\{1,2, \cdots \}.$ If you are at $1$ go to state $j$ with probability $p_j$ $($$ j=1,2,\cdots$ $) ,$ where these are non-negative numbers adding to $1$. If you are in a state $i>1\ ,$ then go just one step back$,$ that is$,$ to $i-1.$ Discuss the nature of the states and nature of the stationary distribution$.$

Assuming that all $p_j$'s are positive I found that the Markov chain is irreducible with all it's states recurrent$.$ So there is no transient state and hence inessential state in the above Markov chain$.$ While calculating stationary distribution I found a problem$.$ Here it is $:$

Suppose $\pi = (\pi_1 , \pi_2 , \cdots )$ be the stationary distribution of the above Markov chain$.$ Then I found that $\pi_2 = (1-p_1) \pi_1, \pi_3 = (1-p_1-p_2) \pi_1, \cdots$ Since $\sum\limits_{i=1}^{\infty} \pi_i = 1$ so $\pi_1 \{1 + (1-p_1) + (1-p_1 -p_2) + (1-p_1 -p_2 - p_3) + \cdots \}= 1.$ Now how do I find the sum

$$1 + \sum_{k=1}^{\infty} (1-p_1-p_2 -p_3 - \cdots - p_k)\ ?$$

Or in other words

$$\sum\limits_{k=1}^{\infty} \sum\limits_{j=k}^{\infty} p_j\ ?$$

This gives us $\pi_1$ and consequently all the $\pi_i$'s$.$ where

$$\pi_i = \frac {\sum\limits_{k=i}^{\infty} p_k} {\sum\limits_{k=1}^{\infty} \sum\limits_{j=k}^{\infty} p_j}.$$

for $i=1,2,\cdots.$

Please help me in this regard. Thank you very much.

$\endgroup$
5
  • $\begingroup$ I get $\sum_{j=1}^\infty{jp_j}$ The value of this will depend on the $p_j$ of course. I think you've solved the problem already. $\endgroup$
    – saulspatz
    Dec 5, 2018 at 15:02
  • $\begingroup$ Whose value is $\sum_{j=1}^{\infty} jp_j$? $\endgroup$
    – little o
    Dec 5, 2018 at 15:12
  • $\begingroup$ $1+(1-p_1)+(1-p_1-p_2)+(1-p_1-p_2-p_3)+\dots=\sum_{j=1}^{\infty} jp_j$ $\endgroup$
    – saulspatz
    Dec 5, 2018 at 15:17
  • $\begingroup$ How? I got $$\sum\limits_{k=1}^{\infty} \sum\limits_{j=k}^{\infty} p_j.$$ You may check. How do you get your expression? Would you please share @saulspatz? $\endgroup$
    – little o
    Dec 5, 2018 at 15:29
  • $\begingroup$ Yeah I have understood. Anyway same expression. Actually I love multiple sums.😀 $\endgroup$
    – little o
    Dec 5, 2018 at 15:30

1 Answer 1

Reset to default
1
$\begingroup$

$$\begin{align} \sum_{j=1}^\infty{jp_j}&=\sum_{j=1}^\infty{p_j}+\sum_{j=2}^\infty{(j-1)p_j}\\ &=1+\sum_{j=2}^\infty{p_j}+\sum_{j=3}^\infty{(j-2)p_j}\\ &=1+(1-p_1)+\sum_{j=3}^\infty{p_j}+\sum_{j=4}^\infty{(j-3)p_j}\\ &\vdots \end{align}$$

EDIT

This is true if the given sum converges. As discussed in the accepted answer to this question, it need not.

$\endgroup$
13
  • $\begingroup$ I have understood it already. Thanks for your answer. $\endgroup$
    – little o
    Dec 5, 2018 at 15:37
  • $\begingroup$ @Dbchatto67 Right, I'd just finished typing it when I saw your comment, so I went ahead and posted it. An interesting point is that this problem shows if $p_j\ge0$ and $\sum{p_j}=1,$ then $\sum{jp_j}$ converges. Is there a more direct proof? $\endgroup$
    – saulspatz
    Dec 5, 2018 at 15:44
  • $\begingroup$ Hmm. Good observation. May be there are direct approaches but I am not familiar with those. Sorry for that. Do you know some of them? Then please don't forget to share your thoughts. That will also be helpful for me. $\endgroup$
    – little o
    Dec 5, 2018 at 15:54
  • $\begingroup$ @Dbchatto67 There must be an error here. Look at math.stackexchange.com/questions/3027282/… $\endgroup$
    – saulspatz
    Dec 5, 2018 at 16:30
  • $\begingroup$ Then what is the error lying in the question? $\endgroup$
    – little o
    Dec 5, 2018 at 16:36

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .