# Markov property w.r.t. a countable state space

Background

Let $\left(X_t\right)_{t \in I}$ ($I\subseteq\mathbb R$) be an $E$-valued stochastic process ($E$ being a Polish space with the Borel $\sigma$-algebra $\mathcal{B}\left(E\right)$) equipped with the filtration generated by $X$, $\left(\mathcal F_t\right)_{t\in I}=\left(\sigma\left(X_s\space:\space s\leq t\right)\right)_{t\in I}$. Suppose $E$ is countable.

Question

Why is it the case (as claimed in Klenke, Remark 17.2) that if for all $n\in\mathbb N$, all $s_1<\cdots<s_n<t$ and all $i_1,\dots,i_n,i\in E$ with $\mathbb{P}\left[X_{s_1}=i_1,\dots,X_{s_n}=i_n\right]>0$ we have

$$\mathbb{P}\left[\left.X_t=i\space\right|\space X_{s_1}=i_1,\dots,X_{s_n}=i_n\right]=\mathbb{P}\left[\left.X_t=i\space\right|\space X_{s_n}=i_n\right]$$

then the Markov property applies, namely

$$\forall s\leq t\in I\bullet\mathbb{P}\left[\left.X_t\in A\space\right|\space\mathcal{F}_s\right]=\mathbb{P}\left[\left.X_t\in A\space\right|\space X_s\right]$$

-

Fix $s\leq t$. We can consider that $A=\{i_k\}$, using additivity and the fact that the space is countable. We have to check that $$\forall B\in\mathcal F_s,\quad\int_B\chi_{\{X_t=i_k\}}dP=\int_BE[\chi_{\{X_t=i_k\}}\mid X_s]dP.$$ To see that, note that the equality is true on the finite intersections of sets of the form $\{X_u\in E'\}$, $E'\subset E$ and $u\leq s$, then show that the sets which satisfy the displayed equality is a $\lambda$-system. You will need Dynkin's theorem.
Thank you, Davide. That's a terrific idea, though i'd go about it a little differently: Define $f:=\mathbb{P}\left[\left.X_t=i\space\right|\space\mathcal{F}_s\right]$. We need to show: $\mathbb{P}\left[\left.X_t=i\space\right|\space X_s\right]=f\space\space a.s.$ In order to do so, we need to prove two propositions: (a) $f$ is $\sigma\left(X_s\right)$-measurable, (b) $\forall B\in \sigma\left(X_s\right)\bullet\int_B f d\mathbb{P}=\mathbb{P}\left(\{X_t=i\}\cap B\right)$. Since $\sigma\left(X_s\right)\subseteq\mathcal{F}_s$, (b) is immediate. [to be continued...] – Evan Aad Aug 9 '12 at 20:55
[... continued] To show (a) it suffices to consider $B$s that are finite intersections of sets of the form $\{X_r=j\}$, $r\leq s$, $j\in E$, since they constitue a $\pi$-system that generates $\mathcal{F}_s$. – Evan Aad Aug 9 '12 at 20:56