Show that positive recurrence is a class property

Question

So, I want to prove that null recurrence and positive recurrence are class properties for Markov chains. It suffices to show that if one state is positive recurrent, all states are positive recurrent. My lecture notes tell me that this can be proven in a similar way to how it is proven that recurrence / transience are class properties, but I don't think that this is entirely correct.

Say we have $i$, $j$ - two states in a communicating class, and $i$ is positive recurrent, i.e. $\mathbb{E}_i(\inf \{n \geq 1: X_n = i\}) < \infty$. I want to show that $y$ is also positive recurrent, and for that I want to bound $\mathbb{E}_j(\inf \{n \geq 1: X_n = j\}) = \sum_n n p_{j,j}^n \prod_k (1 - p_{j,j}^k)$ (where $p_{j,j}^n$ is the probability that we return to step $j$ within $n$ steps) somehow. However, this does not seem to be easy - we cannot consider only paths that go through state $i$ (as it does not necessarily increase or decrease the sum).

Could someone please help me with that? Thank you!

Answer 1

The proof is shorter if we use other definitions of positive recurrence, but it is also instructive to see how one uses the expected return time definition.

Let $T_i:=\inf \{n \geq 1 : X_n = i\}$, so the hypothesis is $E_i(T_i)<\infty$.

Define $T_{i,0}:=0$ and inductively $T_{i,k+1}:=\inf \{n > T_{i,k }: X_n = i\},$ so that for the chain started at $i$, the increments $\Delta_k:=T_{i,k}-T_{i,k-1}$ are i.i.d. for $k=1,2,\dots$ Note that $\Delta_1=T_{i,1}:=T_i$.

Write $\psi:= P_i(T_j<T_i)$, which is positive since $i$ and $j$ are in the same communicating class. Then $E_i(T_i) \ge \psi \cdot E_j(T_i)$, so $E_j(T_i)<\infty. \quad (*)$

Moreover, for the chain started at $i$, the stopping time $\tau:=\inf \{k \geq 1 : T_j <T_{i,k}\}$ has a geometric distribution with parameter $\psi$. Thus by Wald's lemma [1], $$E_i[T_{i,\tau}]=E_i\Bigl[\sum_{k=1}^\tau \Delta_k\Bigr]=E_i(\tau)\cdot E_i(\Delta_1)=E_i(\tau)\cdot E_i(T_i)<\infty\,.$$ Since $T_j<T_{i,\tau}$ for the chain started at $i$, we infer that $E_i(T_j)<\infty.$ In conjunction with $(*)$, this yields $$E_j(T_j) \le E_j(T_i)+E_i(T_j) <\infty \,.$$

[1] https://en.wikipedia.org/wiki/Wald%27s_equation

Answer 2

If state j has period d, we let $\pi_j=\lim\limits_{n\rightarrow\infty}P^{nd}_{jj}$ .

Lemma: If j is recurrent, then j is positive recurrent if and only if $\pi_j>0$.

We denote ${N(t)}$ as the number of times j arrived before time t. We find that ${N(t)}$ is a renewal process. According to Blackwell's Theorem, we obtain that $$\pi_j=\lim\limits_{n\rightarrow\infty}P^{nd}_{jj}=\lim\limits_{n\rightarrow\infty}d\frac{m((n+1)d)-m(nd)}{d}=\frac{d}{\mu_{j}},where \enspace \mu_j=\sum^{\infty}_{k=1}kf^k_{ij}$$

$\Rightarrow$j is positive recurrent $\Leftrightarrow \mu_j<\infty \Leftrightarrow \pi_j>0$

If i and j communicate, then there exists $k_{ij}$ and $k_{ji}$ such that $P^{k_{ij}}_{ij}>0,P^{k_{ji}}_{ji}>0$. $$P^{k_{ji}+k_{ij}}_{jj}\geq P^{k_{ji}}_{ji}P^{{k_{ij}}}_{ij}>0 $$ $$\Rightarrow d|k_{ji}+k_{ij}$$$$\Rightarrow k_{ji}+k_{ij}=md,m\in \mathcal{Z}$$$$\pi_i=\lim\limits_{n\rightarrow\infty}P^{nd}_{ii}\geq \lim\limits_{n\rightarrow\infty}P^{k_{ij}}_{ij}P^{nd-k_{ij}-k_{ji}}_{jj}P^{k_{ji}}_{ij}=\lim\limits_{n\rightarrow\infty}P^{k_{ij}}_{ij}P^{(n-m)d}_{jj}P^{k_{ji}}_{ij}=P^{k_{ij}}_{ij} \pi_{j}P^{k_{ji}}_{ij}>0$$ $\Rightarrow$i is positive recurrent.