# Show that positive recurrence is a class property

So, I want to prove that null recurrence and positive recurrence are class properties for Markov chains. It suffices to show that if one state is positive recurrent, all states are positive recurrent. My lecture notes tell me that this can be proven in a similar way to how it is proven that recurrence / transience are class properties, but I don't think that this is entirely correct.

Say we have $$i$$, $$j$$ - two states in a communicating class, and $$i$$ is positive recurrent, i.e. $$\mathbb{E}_i(\inf \{n \geq 1: X_n = i\}) < \infty$$. I want to show that $$y$$ is also positive recurrent, and for that I want to bound $$\mathbb{E}_j(\inf \{n \geq 1: X_n = j\}) = \sum_n n p_{j,j}^n \prod_k (1 - p_{j,j}^k)$$ (where $$p_{j,j}^n$$ is the probability that we return to step $$j$$ within $$n$$ steps) somehow. However, this does not seem to be easy - we cannot consider only paths that go through state $$i$$ (as it does not necessarily increase or decrease the sum).

The proof is shorter if we use other definitions of positive recurrence, but it is also instructive to see how one uses the expected return time definition.

Let $$T_i:=\inf \{n \geq 1 : X_n = i\}$$, so the hypothesis is $$E_i(T_i)<\infty$$.

Define $$T_{i,0}:=0$$ and inductively $$T_{i,k+1}:=\inf \{n > T_{i,k }: X_n = i\},$$ so that for the chain started at $$i$$, the increments $$\Delta_k:=T_{i,k}-T_{i,k-1}$$ are i.i.d. for $$k=1,2,\dots$$ Note that $$\Delta_1=T_{i,1}:=T_i$$.

Write $$\psi:= P_i(T_j, which is positive since $$i$$ and $$j$$ are in the same communicating class. Then $$E_i(T_i) \ge \psi \cdot E_j(T_i)$$, so $$E_j(T_i)<\infty. \quad (*)$$

Moreover, for the chain started at $$i$$, the stopping time $$\tau:=\inf \{k \geq 1 : T_j has a geometric distribution with parameter $$\psi$$. Thus by Wald's lemma [1], $$E_i[T_{i,\tau}]=E_i\Bigl[\sum_{k=1}^\tau \Delta_k\Bigr]=E_i(\tau)\cdot E_i(\Delta_1)=E_i(\tau)\cdot E_i(T_i)<\infty\,.$$ Since $$T_j for the chain started at $$i$$, we infer that $$E_i(T_j)<\infty.$$ In conjunction with $$(*)$$, this yields $$E_j(T_j) \le E_j(T_i)+E_i(T_j) <\infty \,.$$

• Great answer! You can make it more self contained by not using Wald. $$E_i \left [ \sum_{k=1}^\tau \Delta_k \right] = \sum_{s=1}^\infty \sum_{k=1}^s E_i[\Delta_k | \tau =s] P_i(\tau=s) = E_i[\tau] E_i[T_i]$$ where we use the strong Markov property. Commented Jan 25, 2023 at 15:41
• @NoahMarshall "Everything should be made as simple as possible, but not simpler." The $\Delta_k$ are dependent on $\tau$. Commented Jan 28, 2023 at 22:52
• Why is that $E_i(T_i)\geq\psi\cdot E_j(T_i)$? Commented May 2 at 3:50
• @PorkingBun Starting from $i$, the hitting time $T_i$ is at least the indicator of the event that $j$ is visited before returning to $i$, times the hitting time of $i$ starting from $j$. Now take expectation of this inequality, using the strong Markov property at time $T_j$. Commented May 3 at 7:39
• @PorkingBun I think the crucial observation is that for all $k$ $(S<T)\cap(T=k)=(S< k)\cap (T=k)\in \mathcal{F}_k$. See also math.stackexchange.com/questions/4936083/…. Commented Jun 22 at 8:45

If state j has period d, we let $$\pi_j=\lim\limits_{n\rightarrow\infty}P^{nd}_{jj}$$ .

Lemma: If j is recurrent, then j is positive recurrent if and only if $$\pi_j>0$$.

We denote $${N(t)}$$ as the number of times j arrived before time t. We find that $${N(t)}$$ is a renewal process. According to Blackwell's Theorem, we obtain that $$\pi_j=\lim\limits_{n\rightarrow\infty}P^{nd}_{jj}=\lim\limits_{n\rightarrow\infty}d\frac{m((n+1)d)-m(nd)}{d}=\frac{d}{\mu_{j}},where \enspace \mu_j=\sum^{\infty}_{k=1}kf^k_{ij}$$

$$\Rightarrow$$j is positive recurrent $$\Leftrightarrow \mu_j<\infty \Leftrightarrow \pi_j>0$$

If i and j communicate, then there exists $$k_{ij}$$ and $$k_{ji}$$ such that $$P^{k_{ij}}_{ij}>0,P^{k_{ji}}_{ji}>0$$. $$P^{k_{ji}+k_{ij}}_{jj}\geq P^{k_{ji}}_{ji}P^{{k_{ij}}}_{ij}>0$$ $$\Rightarrow d|k_{ji}+k_{ij}$$$$\Rightarrow k_{ji}+k_{ij}=md,m\in \mathcal{Z}$$$$\pi_i=\lim\limits_{n\rightarrow\infty}P^{nd}_{ii}\geq \lim\limits_{n\rightarrow\infty}P^{k_{ij}}_{ij}P^{nd-k_{ij}-k_{ji}}_{jj}P^{k_{ji}}_{ij}=\lim\limits_{n\rightarrow\infty}P^{k_{ij}}_{ij}P^{(n-m)d}_{jj}P^{k_{ji}}_{ij}=P^{k_{ij}}_{ij} \pi_{j}P^{k_{ji}}_{ij}>0$$ $$\Rightarrow$$i is positive recurrent.