|
Can anyone help me with this question, maybe by giving a hint. Consider a Markov chain with state space $\{0,1,2....\}$. A sequence of positive numbers $p_1,p_2,...$ is given with $\sum p_i=1$. Whenever the chain reaches 0 it chooses a new state according to the $p_i$. Whenever the chain is at a state other than $0$ it proceeds deterministically one step at a time towards $0$. Under what condition on $p_i$ is the chain positive recurrent? Thanks for your help. |
||||
|
|
|
Consider a state $k$. We are interested in the first return time to this state: $$ T_k = \inf\left\{ n \geqslant 1 \colon X_n = k \mid X_0 = k \right\} $$ The state $k$ is called recurrent if $\mathbb{P}\left(T_k < \infty\right) = 1$, and positive recurrent if $\mathbb{E}\left(T_k\right) < \infty$. With $X_0=k$, the system transitions to the origin in $k$ steps with probability 1, and from there it either jumps to a state $0 \leqslant m < k$ in which case it returns to the origin and we start over again, or it jumps to a state $m \geqslant k$ in which case it returns to the state $k$. The mean first passage time from the origin to the state $k$ is clearly $\mathbb{E}(T_k)-k$. Conditioning on whether the jump occurs to left of $k$ or not: $$ \mathbb{E}\left(T_k\right) - k = \underbrace{\sum_{m=0}^{k-1} m p_m + \left(\mathbb{E}\left(T_k\right) -k\right) \sum_{m=0}^{k-1} p_m}_{ \text{transition to } m < k} + \underbrace{\sum_{m=k}^\infty (m-k) p_m}_{\text{transition to } m\geqslant k} $$ giving: $$ \mathbb{E}\left(T_k\right) \left(1-\sum_{m=0}^{k-1} p_m\right) = \sum_{m=0}^\infty m p_m \qquad \therefore \quad \mathbb{E}\left(T_k\right) = \frac{\sum_{m=0}^\infty m p_m}{\sum_{m=k}^\infty p_m} $$ Implying that any state $k$ is positive recurrent provided $$ \sum_{m=0}^\infty m p_m < \infty $$ Here we assume that for all $k \geqslant 0$, $\sum_{m=k}^\infty p_m > 0$. |
|||||||||||||
|
