# Markov Chains and their invariants: exchanging limits

I've been told the following result:

Prop. Let $$X$$ be a markov chain on $$S$$ (not necessarily finite) and transistion matrix $$P=(p_{i,j})_{i,j\in S}$$. Then if for some $$i\in S$$ we have that for all $$j\in S$$, $$p^{(n)}_{i,j}\rightarrow \pi_j$$ as $$n\rightarrow \infty$$, then $$(\pi_j)_{j\in S}$$ in an invariant measure (i.e. $$\pi P=P$$). Moreover if $$S$$ is finite then it's not only a measure but a probability distribution.

Well, if $$S$$ is finite then everything is easy to me, but i cannot understand the case in which $$S$$ is not finite. Indeed the argument that i came up with is the following:

$$\pi_j=\displaystyle \lim_{n\rightarrow \infty} p_{i,j}^{(n)}=\lim_{n\rightarrow \infty} \sum_{k\in S}p_{i,k}^{(n-1)}p_{k,j}=\sum_{k\in S} \lim_{n\rightarrow \infty} p_{i,k}^{(n-1)}p_{k,j}=\sum_{k\in S} \pi_{k}p_{k,j}$$

The problem here is the third equality; if $$S$$ in not finite then i'm exchanging the limit with a series and i don't know how to justify it. I also think that in this setting monotone/dominated convergence theorems cannot help beacause i'm not able to see that series as integral w.r.t some measure on which i can apply those theorems.

So is this result correct but require a different proof? Or is the result not correct for $$S$$ countable? Or is the limit exchange i pointed out correct and it's just me that i can't understand it?

• That sum is just an integral with respect to the counting measure on $S$. – Rhys Steele Jun 12 at 9:26
• ok, i can see this but using the counting measure i cannot apply dominated convergence or monotone convergence (the sequence is not monotone and i can't find an integrable dominating function) – StabiloBoss Jun 12 at 10:09
• All terms are nonnegative. So if you sum over only finitely many terms but then take a limit of the result you get $\pi\geq\pi P$. – Michael Jun 12 at 11:02
• ok @Michael that's correct, but still not enough to have the equality that is needed. – StabiloBoss Jun 12 at 12:03
• You can then prove equality: Imagine that one of the constraints is strict. But now sum over all $j$. Equivalently, multiply both sides on the right by the all-1 vector. You may want Fubini-Tonelli to justify switching sums of nonnegative terms. – Michael Jun 12 at 14:28

The result is correct. Let $$S = \{1, 2, 3, ...\}$$ be countably infinite.
1) First prove that $$\sum_{j \in S} \pi_j \leq 1$$
2) Next prove that $$\pi \geq \pi P$$ (via the hint in my comment above), equivalently:
$$\pi_j \geq \sum_{k \in S} \pi_k P_{kj} \quad \forall j \in S \quad (Eq. 1)$$
3) Suppose (Eq. 1) holds with strict inequality for at least one $$j\in S$$. Summing over all $$j \in S$$ gives $$\sum_{j \in S} \pi_j > \sum_{j \in S} \sum_{k \in S} \pi_k P_{kj}$$ where we have used the fact that the sums are finite to eliminate the $$\infty \geq \infty$$ case. Switch the sums of the nonnegative terms (Fubini-Tonelli) to reach a contradiction.
• PS: I suppose (Eq. 1) could be justified by Fatou's lemma but it is more basic to justify via $$P_{ij}^{(n)} = \sum_{k=1}^{\infty} P_{ik}^{(n-1)}P_{kj} \geq \sum_{k=1}^m P_{ik}^{(n-1)}P_{kj}$$ – Michael Jun 12 at 14:54