# Markov chain restricted to a subset

Suppose $S$ is a countable state space and $F \subset S$. Let $X_n$ be an irreducible, recurrent Markov chain with transition probabilities $p(x,y)$. Now if $T_n$ is the $n^{th}$ time the chain hits the set $F$, then I have shown that $F_{T_n}$ defines a Markov chain, adapted to the filtration $F_{T_n}$. This followed from the strong Markov property.

Now I also showed that the new Markov chain has transition probabilities $q(a,b) = P^a(T_1 = b)$ and in fact $q^n(a,b) = P^a(T_n = b)$. I managed to show this new chain is irreducible using the fact that $\{ X_m = b \} = \cup_{j=1}^m \{ X_{T_j} = b, T_j = m\}$ a disjoint union. If the left side has positive probability by irreducibility, then a set on the right must have positive probability.

However I don't know how to show the new chain is recurrent. Since it is irreducible, it is enough to show one state is recurrent.

• Remember that by definition, every return time of $X_0,X_1,\ldots$ to a state in $F$ is an element of $\{0,T_1,T_2,\ldots\}$. Commented Mar 30, 2018 at 3:14

Fix $x \in F$ and consider any fixed path $X_n$ which hits $x$ infinitely often. Consider the subpath $X_{n_k}$ at the times where $X_n$ hits $x$. Because $x \in F$, each of the $n_k$ is some $T_j$. So $X_{T_n}$ hit $x$ infinitely often. But $x$ was an arbitrary element of $F$ and the original path had the required property with probability $1$...