I've been told the following result:

Prop. Let $X$ be a markov chain on $S$ (not necessarily finite) and transistion matrix $P=(p_{i,j})_{i,j\in S}$. Then if for some $i\in S$ we have that for all $j\in S$, $p^{(n)}_{i,j}\rightarrow \pi_j$ as $n\rightarrow \infty$, then $(\pi_j)_{j\in S}$ in an invariant measure (i.e. $\pi P=P$). Moreover if $S$ is finite then it's not only a measure but a probability distribution.

Well, if $S$ is finite then everything is easy to me, but i cannot understand the case in which $S$ is not finite. Indeed the argument that i came up with is the following:

$$\pi_j=\displaystyle \lim_{n\rightarrow \infty} p_{i,j}^{(n)}=\lim_{n\rightarrow \infty} \sum_{k\in S}p_{i,k}^{(n-1)}p_{k,j}=\sum_{k\in S} \lim_{n\rightarrow \infty} p_{i,k}^{(n-1)}p_{k,j}=\sum_{k\in S} \pi_{k}p_{k,j} $$

The problem here is the third equality; if $S$ in not finite then i'm exchanging the limit with a series and i don't know how to justify it. I also think that in this setting monotone/dominated convergence theorems cannot help beacause i'm not able to see that series as integral w.r.t some measure on which i can apply those theorems.

So is this result correct but require a different proof? Or is the result not correct for $S$ countable? Or is the limit exchange i pointed out correct and it's just me that i can't understand it?

  • $\begingroup$ That sum is just an integral with respect to the counting measure on $S$. $\endgroup$ – Rhys Steele Jun 12 at 9:26
  • $\begingroup$ ok, i can see this but using the counting measure i cannot apply dominated convergence or monotone convergence (the sequence is not monotone and i can't find an integrable dominating function) $\endgroup$ – StabiloBoss Jun 12 at 10:09
  • $\begingroup$ All terms are nonnegative. So if you sum over only finitely many terms but then take a limit of the result you get $\pi\geq\pi P$. $\endgroup$ – Michael Jun 12 at 11:02
  • $\begingroup$ ok @Michael that's correct, but still not enough to have the equality that is needed. $\endgroup$ – StabiloBoss Jun 12 at 12:03
  • $\begingroup$ You can then prove equality: Imagine that one of the constraints is strict. But now sum over all $j$. Equivalently, multiply both sides on the right by the all-1 vector. You may want Fubini-Tonelli to justify switching sums of nonnegative terms. $\endgroup$ – Michael Jun 12 at 14:28

The result is correct. Let $S = \{1, 2, 3, ...\}$ be countably infinite.

1) First prove that $$ \sum_{j \in S} \pi_j \leq 1$$

2) Next prove that $\pi \geq \pi P$ (via the hint in my comment above), equivalently:

$$ \pi_j \geq \sum_{k \in S} \pi_k P_{kj} \quad \forall j \in S \quad (Eq. 1)$$

3) Suppose (Eq. 1) holds with strict inequality for at least one $j\in S$. Summing over all $j \in S$ gives $$ \sum_{j \in S} \pi_j > \sum_{j \in S} \sum_{k \in S} \pi_k P_{kj}$$ where we have used the fact that the sums are finite to eliminate the $\infty \geq \infty$ case. Switch the sums of the nonnegative terms (Fubini-Tonelli) to reach a contradiction.

  • $\begingroup$ PS: I suppose (Eq. 1) could be justified by Fatou's lemma but it is more basic to justify via $$ P_{ij}^{(n)} = \sum_{k=1}^{\infty} P_{ik}^{(n-1)}P_{kj} \geq \sum_{k=1}^m P_{ik}^{(n-1)}P_{kj}$$ $\endgroup$ – Michael Jun 12 at 14:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.